1
|
Sidhu DM, Athanasopoulou A, Archer SL, Czarnecki N, Curtin S, Pexman PM. The maluma/takete effect is late: No longitudinal evidence for shape sound symbolism in the first year. PLoS One 2023; 18:e0287831. [PMID: 37943758 PMCID: PMC10635456 DOI: 10.1371/journal.pone.0287831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 06/14/2023] [Indexed: 11/12/2023] Open
Abstract
The maluma/takete effect refers to an association between certain language sounds (e.g., /m/ and /o/) and round shapes, and other language sounds (e.g., /t/ and /i/) and spiky shapes. This is an example of sound symbolism and stands in opposition to arbitrariness of language. It is still unknown when sensitivity to sound symbolism emerges. In the present series of studies, we first confirmed that the classic maluma/takete effect would be observed in adults using our novel 3-D object stimuli (Experiments 1a and 1b). We then conducted the first longitudinal test of the maluma/takete effect, testing infants at 4-, 8- and 12-months of age (Experiment 2). Sensitivity to sound symbolism was measured with a looking time preference task, in which infants were shown images of a round and a spiky 3-D object while hearing either a round- or spiky-sounding nonword. We did not detect a significant difference in looking time based on nonword type. We also collected a series of individual difference measures including measures of vocabulary, movement ability and babbling. Analyses of these measures revealed that 12-month olds who babbled more showed a greater sensitivity to sound symbolism. Finally, in Experiment 3, we had parents take home round or spiky 3-D printed objects, to present to 7- to 8-month-old infants paired with either congruent or incongruent nonwords. This language experience had no effect on subsequent measures of sound symbolism sensitivity. Taken together these studies demonstrate that sound symbolism is elusive in the first year, and shed light on the mechanisms that may contribute to its eventual emergence.
Collapse
Affiliation(s)
- David M. Sidhu
- Department of Psychology, Carleton University, Ottawa, Canada
| | - Angeliki Athanasopoulou
- School of Languages, Linguistics, Literatures, and Cultures, University of Calgary, Calgary, Canada
| | | | | | - Suzanne Curtin
- Department of Child and Youth Studies, Brock University, St. Catharines, Canada
| | - Penny M. Pexman
- Department of Psychology, University of Calgary, Calgary, Canada
| |
Collapse
|
2
|
Barany DA, Lacey S, Matthews KL, Nygaard LC, Sathian K. Neural basis of sound-symbolic pseudoword-shape correspondences. Neuropsychologia 2023; 188:108657. [PMID: 37543139 PMCID: PMC10529692 DOI: 10.1016/j.neuropsychologia.2023.108657] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 06/23/2023] [Accepted: 08/02/2023] [Indexed: 08/07/2023]
Abstract
Non-arbitrary mapping between the sound of a word and its meaning, termed sound symbolism, is commonly studied through crossmodal correspondences between sounds and visual shapes, e.g., auditory pseudowords, like 'mohloh' and 'kehteh', are matched to rounded and pointed visual shapes, respectively. Here, we used functional magnetic resonance imaging (fMRI) during a crossmodal matching task to investigate the hypotheses that sound symbolism (1) involves language processing; (2) depends on multisensory integration; (3) reflects embodiment of speech in hand movements. These hypotheses lead to corresponding neuroanatomical predictions of crossmodal congruency effects in (1) the language network; (2) areas mediating multisensory processing, including visual and auditory cortex; (3) regions responsible for sensorimotor control of the hand and mouth. Right-handed participants (n = 22) encountered audiovisual stimuli comprising a simultaneously presented visual shape (rounded or pointed) and an auditory pseudoword ('mohloh' or 'kehteh') and indicated via a right-hand keypress whether the stimuli matched or not. Reaction times were faster for congruent than incongruent stimuli. Univariate analysis showed that activity was greater for the congruent compared to the incongruent condition in the left primary and association auditory cortex, and left anterior fusiform/parahippocampal gyri. Multivoxel pattern analysis revealed higher classification accuracy for the audiovisual stimuli when congruent than when incongruent, in the pars opercularis of the left inferior frontal (Broca's area), the left supramarginal, and the right mid-occipital gyri. These findings, considered in relation to the neuroanatomical predictions, support the first two hypotheses and suggest that sound symbolism involves both language processing and multisensory integration.
Collapse
Affiliation(s)
- Deborah A Barany
- Department of Kinesiology, University of Georgia and Augusta University/University of Georgia Medical Partnership, Athens, GA, 30602, USA
| | - Simon Lacey
- Department of Neurology, Penn State College of Medicine, Hershey, PA, 17033-0859, USA; Department of Neural & Behavioral Sciences, Penn State College of Medicine, Hershey, PA, 17033-0859, USA; Department of Psychology, Penn State College of Liberal Arts, University Park, PA, 16802, USA
| | - Kaitlyn L Matthews
- Department of Psychology, Emory University, Atlanta, GA, 30322, USA; Present address: Department of Psychological & Brain Sciences, Washington University in St. Louis, St. Louis, MO, 63130, USA
| | - Lynne C Nygaard
- Department of Psychology, Emory University, Atlanta, GA, 30322, USA
| | - K Sathian
- Department of Neurology, Penn State College of Medicine, Hershey, PA, 17033-0859, USA; Department of Neural & Behavioral Sciences, Penn State College of Medicine, Hershey, PA, 17033-0859, USA; Department of Psychology, Penn State College of Liberal Arts, University Park, PA, 16802, USA.
| |
Collapse
|
3
|
Sidhu DM, Vigliocco G. I don't see what you're saying: The maluma/takete effect does not depend on the visual appearance of phonemes as they are articulated. Psychon Bull Rev 2023; 30:1521-1529. [PMID: 36520277 PMCID: PMC10482773 DOI: 10.3758/s13423-022-02224-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/16/2022] [Indexed: 12/23/2022]
Abstract
In contrast to the principle of arbitrariness, recent work has shown that language can iconically depict referents being talked about. One such example is the maluma/takete effect: an association between certain phonemes (e.g., those in maluma) and round shapes, and other phonemes (e.g., those in takete and spiky shapes). An open question has been whether this association is crossmodal (arising from phonemes' sound or kinesthetics) or unimodal (arising from phonemes' visual appearance). In the latter case, individuals may associate a person's rounded lips as they pronounce the /u/ in maluma with round shapes. We examined this hypothesis by having participants pair nonwords with shapes in either an audio-only condition (they only heard nonwords) or an audiovisual condition (they both heard nonwords and saw them articulated). We found no evidence that seeing nonwords articulated enhanced the maluma/takete effect. In fact, there was evidence that it decreased it in some cases. This was confirmed with a Bayesian analysis. These results eliminate a plausible explanation for the maluma/takete effect, as an instance of visual matching. We discuss the alternate possibility that it involves crossmodal associations.
Collapse
Affiliation(s)
- David M. Sidhu
- Department of Psychology, University College London, London, UK
- Department of Psychology, Carleton University, Ottawa, Ontario Canada
| | | |
Collapse
|
4
|
Resolving the bouba-kiki effect enigma by rooting iconic sound symbolism in physical properties of round and spiky objects. Sci Rep 2022; 12:19172. [PMID: 36357511 PMCID: PMC9649795 DOI: 10.1038/s41598-022-23623-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Accepted: 11/02/2022] [Indexed: 11/11/2022] Open
Abstract
The "bouba-kiki effect", where "bouba" is perceived round and "kiki" spiky, remains a puzzling enigma. We solve it by combining mathematical findings largely unknown in the field, with computational models and novel experimental evidence. We reveal that this effect relies on two acoustic cues: spectral balance and temporal continuity. We demonstrate that it is not speech-specific but rather rooted in physical properties of objects, creating audiovisual regularities in the environment. Round items are mathematically bound to produce, when hitting or rolling on a surface, lower-frequency spectra and more continuous sounds than same-size spiky objects. Finally, we show that adults are sensitive to such regularities. Hence, intuitive physics impacts language perception and possibly language acquisition and evolution too.
Collapse
|
5
|
Gravener M, Lacey S, Sathian K. Perceptual Dissimilarity Analysis Distinguishes Grapheme-Color Synesthetes from Nonsynesthetes. Cogn Sci 2022; 46:e13189. [PMID: 36036150 DOI: 10.1111/cogs.13189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 06/20/2022] [Accepted: 08/05/2022] [Indexed: 11/29/2022]
Abstract
Synesthetes can be distinguished from nonsynesthetes on a variety of experimental tasks because their concurrent synesthetic experiences can affect task performance if these experiences match or conflict with some aspect of the stimulus. Here, we tested grapheme-color synesthetes and nonsynesthetic control participants using a novel perceptual similarity task to assess whether synesthetes' concurrent color experiences influence perceived grapheme similarity. Participants iteratively arranged graphemes and, separately, their associated synesthetic colors in a display, such that similar items were placed close together and dissimilar items further apart. The resulting relative inter-item distances were used to calculate the pair-wise (dis)similarity between items in the set, and thence to create separate perceptual representational dissimilarity matrices (RDMs) for graphemes and colors, on an individual basis. On the assumption that synesthetes' similarity judgments for graphemes would be influenced by their concurrent color experiences, we predicted that grapheme and color RDMs would be more strongly correlated for synesthetes than nonsynesthetes. We found that the mean grapheme-color RDM correlation was indeed significantly higher in synesthetes than nonsynesthetes; in addition, synesthetes' grapheme-color RDM correlations were more likely to be individually statistically significant, even after correction for multiple tests, than those of nonsynesthetes. Importantly, synesthetes' grapheme-color RDM correlations were scaled with the consistency of their grapheme-color associations as measured by their Synesthesia Battery (SB) scores. By contrast, the relationship between SB scores and grapheme-color RDM correlations for nonsynesthetes was not significant. Thus, dissimilarity analysis quantitatively distinguished synesthetes from nonsynesthetes, in a way that meaningfully reflects a key aspect of synesthetic experience.
Collapse
Affiliation(s)
- Michelle Gravener
- Department of Neurology, Milton S. Hershey Medical Center and Penn State College of Medicine
| | - Simon Lacey
- Department of Neurology, Milton S. Hershey Medical Center and Penn State College of Medicine.,Department of Neural & Behavioral Sciences, Milton S. Hershey Medical Center and Penn State College of Medicine.,Department of Psychology, Milton S. Hershey Medical Center and Penn State College of Medicine
| | - K Sathian
- Department of Neurology, Milton S. Hershey Medical Center and Penn State College of Medicine.,Department of Neural & Behavioral Sciences, Milton S. Hershey Medical Center and Penn State College of Medicine.,Department of Psychology, Milton S. Hershey Medical Center and Penn State College of Medicine
| |
Collapse
|
6
|
Abstract
Roughness is a perceptual attribute typically associated with certain stimuli that are presented in one of the spatial senses. In auditory research, the term is typically used to describe the harsh effects that are induced by particular sound qualities (i.e., dissonance) and human/animal vocalizations (e.g., screams, distress cries). In the tactile domain, roughness is a crucial factor determining the perceptual features of a surface. The same feature can also be ascertained visually, by means of the extraction of pattern features that determine the haptic quality of surfaces, such as grain size and density. By contrast, the term roughness has rarely been applied to the description of those stimuli perceived via the chemical senses. In this review, we take a critical look at the putative meaning(s) of the term roughness, when used in both unisensory and multisensory contexts, in an attempt to answer two key questions: (1) Is the use of the term 'roughness' the same in each modality when considered individually? and (2) Do crossmodal correspondences involving roughness match distinct perceptual features or (at least on certain occasions) do they merely pick-up on an amodal property? We start by examining the use of the term in the auditory domain. Next, we summarize the ways in which the term roughness has been used in the literature on tactile and visual perception, and in the domain of olfaction and gustation. Then, we move on to the crossmodal context, reviewing the literature on the perception of roughness in the audiovisual, audiotactile, and auditory-gustatory/olfactory domains. Finally, we highlight some limitations of the reviewed literature and we outline a number of key directions for future empirical research in roughness perception.
Collapse
|
7
|
Motoki K, Pathak A, Spence C. Tasting prosody: Crossmodal correspondences between voice quality and basic tastes. Food Qual Prefer 2022. [DOI: 10.1016/j.foodqual.2022.104621] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
8
|
Ćwiek A, Fuchs S, Draxler C, Asu EL, Dediu D, Hiovain K, Kawahara S, Koutalidis S, Krifka M, Lippus P, Lupyan G, Oh GE, Paul J, Petrone C, Ridouane R, Reiter S, Schümchen N, Szalontai Á, Ünal-Logacev Ö, Zeller J, Perlman M, Winter B. The bouba/kiki effect is robust across cultures and writing systems. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200390. [PMID: 34775818 PMCID: PMC8591387 DOI: 10.1098/rstb.2020.0390] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Accepted: 09/10/2021] [Indexed: 01/06/2023] Open
Abstract
The bouba/kiki effect-the association of the nonce word bouba with a round shape and kiki with a spiky shape-is a type of correspondence between speech sounds and visual properties with potentially deep implications for the evolution of spoken language. However, there is debate over the robustness of the effect across cultures and the influence of orthography. We report an online experiment that tested the bouba/kiki effect across speakers of 25 languages representing nine language families and 10 writing systems. Overall, we found strong evidence for the effect across languages, with bouba eliciting more congruent responses than kiki. Participants who spoke languages with Roman scripts were only marginally more likely to show the effect, and analysis of the orthographic shape of the words in different scripts showed that the effect was no stronger for scripts that use rounder forms for bouba and spikier forms for kiki. These results confirm that the bouba/kiki phenomenon is rooted in crossmodal correspondence between aspects of the voice and visual shape, largely independent of orthography. They provide the strongest demonstration to date that the bouba/kiki effect is robust across cultures and writing systems. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.
Collapse
Affiliation(s)
- Aleksandra Ćwiek
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
- Institut für deutsche Sprache und Linguistik, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
| | - Susanne Fuchs
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
| | - Christoph Draxler
- Institute of Phonetics and Speech Processing, Ludwig Maximilian University, 80799 Munich, Germany
| | - Eva Liina Asu
- Institute of Estonian and General Linguistics, University of Tartu, 50090 Tartu, Estonia
| | - Dan Dediu
- Laboratoire Dynamique Du Langage UMR 5596, Université Lumière Lyon 2, 69363 Lyon, France
| | - Katri Hiovain
- Department of Digital Humanities, University of Helsinki, 00014 Helsinki, Finland
| | - Shigeto Kawahara
- The Institute of Cultural and Linguistic Studies, Keio University, Mita Minatoku, Tokyo 108-8345, Japan
| | - Sofia Koutalidis
- Faculty of Linguistics and Literary Studies, Bielefeld University, 33615 Bielefeld, Germany
| | - Manfred Krifka
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
- Institut für deutsche Sprache und Linguistik, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
| | - Pärtel Lippus
- Institute of Estonian and General Linguistics, University of Tartu, 50090 Tartu, Estonia
| | - Gary Lupyan
- Department of Psychology, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Grace E. Oh
- Department of English Language and Literature, Konkuk University, Seoul 05029, South Korea
| | - Jing Paul
- Asian Studies Program, Agnes Scott College, Decatur, GA 30030, USA
| | - Caterina Petrone
- Aix-Marseille Université, CNRS, Laboratoire Parole et Langage, UMR 7309, 13100 Aix-en-Provence, France
| | - Rachid Ridouane
- Laboratoire de Phonétique et Phonologie, UMR 7018, CNRS and Sorbonne Nouvelle, 75005 Paris, France
| | - Sabine Reiter
- Depto. de Polonês, Alemão e Letras Clássicas, Universidade Federal do Paraná, 80060-150 Curitiba, Brazil
| | - Nathalie Schümchen
- Department of Language and Communication, University of Southern Denmark, 5230 Odense, Denmark
| | - Ádám Szalontai
- Department of Phonetics, Hungarian Research Centre for Linguistics, Budapest 1068, Hungary
| | - Özlem Ünal-Logacev
- School of Health Sciences, Department of Speech and Language Therapy, Istanbul Medipol University, 34810 Istanbul, Turkey
| | - Jochen Zeller
- School of Arts, Linguistics Discipline, University of KwaZulu-Natal, Durban 4041, South Africa
| | - Marcus Perlman
- Department of English Language and Linguistics, University of Birmingham, Birmingham B15 2TT, UK
| | - Bodo Winter
- Department of English Language and Linguistics, University of Birmingham, Birmingham B15 2TT, UK
| |
Collapse
|
9
|
Lin Z, Wang N, Yan Y, Kambara T. Vowel Length Expands Perceptual and Emotional Evaluations in Written Japanese Sound-Symbolic Words. Behav Sci (Basel) 2021; 11:90. [PMID: 34205574 PMCID: PMC8234476 DOI: 10.3390/bs11060090] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 06/13/2021] [Accepted: 06/16/2021] [Indexed: 11/17/2022] Open
Abstract
In this study, we examined whether vowel length affected the perceptual and emotional evaluations of Japanese sound-symbolic words. The perceptual and emotional features of Japanese sound-symbolic words, which included short and long vowels, were evaluated by 209 native Japanese speakers. The results showed that subjective evaluations of familiarity, visual imageability, auditory imageability, tactile imageability, emotional valence, arousal, and length were significantly higher for sound-symbolic words with long vowels compared to those with short vowels. Additionally, a subjective evaluation of speed was significantly higher for written Japanese sound-symbolic words with short vowels than for those with long vowels. The current findings suggest that vowel length in written Japanese sound-symbolic words increases the perceptually and emotionally subjective evaluations of Japanese sound-symbolic words.
Collapse
Affiliation(s)
| | | | | | - Toshimune Kambara
- Department of Psychology, Graduate School of Humanities and Social Sciences, Hiroshima 7398524, Japan; (Z.L.); (N.W.); (Y.Y.)
| |
Collapse
|
10
|
Akita K. Phonation Types Matter in Sound Symbolism. Cogn Sci 2021; 45:e12982. [PMID: 34018216 PMCID: PMC8244085 DOI: 10.1111/cogs.12982] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Revised: 04/01/2021] [Accepted: 04/03/2021] [Indexed: 11/28/2022]
Abstract
Sound symbolism is a non-arbitrary correspondence between sound and meaning. The majority of studies on sound symbolism have focused on consonants and vowels, and the sound-symbolic properties of suprasegmentals, particularly phonation types, have been largely neglected. This study examines the size and shape symbolism of four phonation types: modal and creaky voices, falsetto, and whisper. Japanese speakers heard 12 novel words (e.g., /íbi/, /ápa/) pronounced with the four types of phonation and rated the size and roundedness/pointedness each of the 48 stimuli seemed to represent on seven-point scales. The results showed that phonation types as well as consonantal and vocalic features influenced the ratings. Creaky voice was associated with larger and more pointed images than modal voice, which was in turn associated with larger and more pointed images than whisper. Falsetto was also associated with roundedness but not with smallness. These results shed new light on the acoustic approaches to sound symbolism and suggest the significance of phonation types and other suprasegmental features in the phenomenon.
Collapse
Affiliation(s)
- Kimi Akita
- Department of English Linguistics, Graduate School of Humanities, Nagoya University
| |
Collapse
|