The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation
March
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation
Alan Taitz 0 1
M. Florencia Assaneo 0 1
Natalia Elisei 1
Mo nica TrÂõpodi 1
Laurent Cohen 1
Jacobo D. Sitt 1
Marcos A. Trevisan 0 1
0 Department of Physics, IFIBA-University of Buenos Aires , Buenos Aires , Argentina , 2 Department of Psychology, New York University, New York, United States of America, 3 Medicine School, University of Buenos Aires , Buenos Aires , Argentina , 4 Department of Linguistics, University of Buenos Aires , Buenos Aires , Argentina , 5 INSERM U1127, Institut du Cerveau et de la Moelle E pinière, Paris, France, 6 CNRS UMR 7225, Institut du Cerveau et de la Moelle E pinière , Paris, France, 7 Sorbonne Universite s, UPMC Univ Paris 06, Paris, France, 8 AP-HP , Groupe Hospitalier Pitie -Salpêtrière, Departament of Neurology , Paris , France
1 Editor: Niels O. Schiller, Leiden University , NETHERLANDS
Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias.
-
Data Availability Statement: All relevant data are
within the paper and its Supporting Information
files.
Funding: The research reported in this work was
partially funded by the Concejo Nacional de
Investigaciones CientÂõficas y TeÂcnicas (CONICET)
to AT and MAT; The University of Buenos Aires
(UBA); ECOS A11S01 scientific collaboration
program (France-Argentina) to MFA, LC, JDS, and
MAT; NIH through R01-DC-012859 to MAT, and
the program ªInvestissements d'avenirº
(ANR-10
Introduction
The arbitrary nature of the linguistic sign has been an idealized notion of modern linguistics
that served to explore the unlimited expressive power of language [
1
]. As research evolved, the
fully arbitrary nature of the link between form and meaning has been called into question,
opening the scientific exploration of this relationship. The results of those investigations show
that nonarbitrary associations are not limited to exceptional cases or to specific word classes: a
IAIHU-06) to JS. The funders had no role in study
design, data collection and analysis, decision to
publish, or preparation of the manuscript.
striking demonstration of this comes from a statistical analysis performed over two-thirds of
the world's languages, revealing that unrelated languages use the same sounds for specific
referents [
2
].
Iconicity is a prominent form of non-arbitrariness, in which different aspects of the form
and the meaning of words are related by perceptuomotor analogies [
3
]. Onomatopoeias are
privileged objects to study iconic properties of spoken words. Just like any other word,
onomatopoeias are embedded in the language and have to adapt to the local phonology, assuming
arbitrary properties. However, they also they tend to maximize the similarity between speech
sounds and the sounds of the actions they represent, preserving parts of the onomatopoeic
structure across languages [
4
].
Beyond sound imitation stand the mimetic words, a more general class used to express
actions where sound is not essential [
5
]. It has been suggested that these words, found in
Japanese, are related to the interaction between the body and the linguistic sound system: mimetic
words use sounds to imitate sensations including body movements, touch, vision, smell, taste,
and sound [
6
]. Interestingly, this kind of words were found in many other languages [7±9].
The concept of ideophone was then coined to characterize 'a word, often onomatopoeic,
which describes a predicate, qualificative or adverb in respect to manner, color, sound, smell,
action, state or intensity' [
10
]. For instance, operations like lengthening and reduplication
tend to evoke repetition and multiplicity, while monosyllabic forms tend to evoke unitary
events [
11
]. Ideophones rise from the rest of the words to depict sensory imagery; beyond the
mere imitation of soun (...truncated)