1
|
Di Stefano N, Ansani A, Schiavio A, Spence C. Prokofiev was (almost) right: A cross-cultural investigation of auditory-conceptual associations in Peter and the Wolf. Psychon Bull Rev 2024; 31:1735-1744. [PMID: 38267741 PMCID: PMC11358347 DOI: 10.3758/s13423-023-02435-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/10/2023] [Indexed: 01/26/2024]
Abstract
Over recent decades, studies investigating cross-modal correspondences have documented the existence of a wide range of consistent cross-modal associations between simple auditory and visual stimuli or dimensions (e.g., pitch-lightness). Far fewer studies have investigated the association between complex and realistic auditory stimuli and visually presented concepts (e.g., musical excerpts-animals). Surprisingly, however, there is little evidence concerning the extent to which these associations are shared across cultures. To address this gap in the literature, two experiments using a set of stimuli based on Prokofiev's symphonic fairy tale Peter and the Wolf are reported. In Experiment 1, 293 participants from several countries and with very different language backgrounds rated the association between the musical excerpts, images and words representing the story's characters (namely, bird, duck, wolf, cat, and grandfather). The results revealed that participants tended to consistently associate the wolf and the bird with the corresponding musical excerpt, while the stimuli of other characters were not consistently matched across participants. Remarkably, neither the participants' cultural background, nor their musical expertise affected the ratings. In Experiment 2, 104 participants were invited to rate each stimulus on eight emotional features. The results revealed that the emotional profiles associated with the music and with the concept of the wolf and the bird were perceived as more consistent between observers than the emotional profiles associated with the music and the concept of the duck, the cat, and the grandpa. Taken together, these findings therefore suggest that certain auditory-conceptual associations are perceived consistently across cultures and may be mediated by emotional associations.
Collapse
Affiliation(s)
- Nicola Di Stefano
- National Research Council, Institute of Cognitive Sciences and Technologies, Rome, Italy.
| | - Alessandro Ansani
- Centre of Excellence in Music, Mind, Body and Brain, Department of Music, Art and Culture Studies, Jyväskylä, Finland
| | - Andrea Schiavio
- University of York, School of Arts and Creative Technologies, York, UK
| | | |
Collapse
|
2
|
Lee MS, Lee GE, Lee SH, Lee JH. Emotional responses of Korean and Chinese women to Hangul phonemes to the gender of an artificial intelligence voice. Front Psychol 2024; 15:1357975. [PMID: 39135868 PMCID: PMC11317464 DOI: 10.3389/fpsyg.2024.1357975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Accepted: 07/04/2024] [Indexed: 08/15/2024] Open
Abstract
Introduction This study aimed to explore the arousal and valence that people experience in response to Hangul phonemes based on the gender of an AI speaker through comparison with Korean and Chinese cultures. Methods To achieve this, 42 Hangul phonemes were used, in a combination of three Korean vowels and 14 Korean consonants, to explore cultural differences in arousal, valence, and the six foundational emotions based on the gender of an AI speaker. A total 136 Korean and Chinese women were recruited and randomly assigned to one of two conditions based on voice gender (man or woman). Results and discussion This study revealed significant differences in arousal levels between Korean and Chinese women when exposed to male voices. Specifically, Chinese women exhibited clear differences in emotional perceptions of male and female voices in response to voiced consonants. These results confirm that arousal and valence may differ with articulation types and vowels due to cultural differences and that voice gender can affect perceived emotions. This principle can be used as evidence for sound symbolism and has practical implications for voice gender and branding in AI applications.
Collapse
Affiliation(s)
- Min-Sun Lee
- Department of Psychology, Chung-Ang University, Seoul, Republic of Korea
| | - Gi-Eun Lee
- Institute of Cultural Diversity Content, Chung-Ang University, Seoul, Republic of Korea
| | - San Ho Lee
- Department of European Language and Cultures, Chung-Ang University, Seoul, Republic of Korea
| | - Jang-Han Lee
- Department of Psychology, Chung-Ang University, Seoul, Republic of Korea
| |
Collapse
|
3
|
Passi A, Arun SP. The Bouba-Kiki effect is predicted by sound properties but not speech properties. Atten Percept Psychophys 2024; 86:976-990. [PMID: 36525201 PMCID: PMC7615921 DOI: 10.3758/s13414-022-02619-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/05/2022] [Indexed: 12/23/2022]
Abstract
Humans robustly associate spiky shapes to words like "Kiki" and round shapes to words like "Bouba." According to a popular explanation, this is because the mouth assumes an angular shape while speaking "Kiki" and a rounded shape for "Bouba." Alternatively, this effect could reflect more general associations between shape and sound that are not specific to mouth shape or articulatory properties of speech. These possibilities can be distinguished using unpronounceable sounds: The mouth-shape hypothesis predicts no Bouba-Kiki effect for these sounds, whereas the generic shape-sound hypothesis predicts a systematic effect. Here, we show that the Bouba-Kiki effect is present for a variety of unpronounceable sounds ranging from reversed words and real object sounds (n = 45 participants) and even pure tones (n = 28). The effect was strongly correlated with the mean frequency of a sound across both spoken and reversed words. The effect was not systematically predicted by subjective ratings of pronounceability or with mouth aspect ratios measured from video. Thus, the Bouba-Kiki effect is explained using simple shape-sound associations rather than using speech properties.
Collapse
Affiliation(s)
- Ananya Passi
- Undergraduate Programme, Indian Institute of Science, Bengaluru, India
| | - S P Arun
- Centre for Neuroscience, Indian Institute of Science, Bengaluru, India.
| |
Collapse
|
4
|
Ogata K, Gakumi R, Hashimoto A, Ushiku Y, Yoshida S. The influence of Bouba- and Kiki-like shape on perceived taste of chocolate pieces. Front Psychol 2023; 14:1170674. [PMID: 37397323 PMCID: PMC10313397 DOI: 10.3389/fpsyg.2023.1170674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 06/01/2023] [Indexed: 07/04/2023] Open
Abstract
In this paper, we present the findings of a study investigating the impact of shape on the taste perception of chocolate. Previous research has explored the influence of various sensory information on taste perception, but there has been little focus on the effect of food shape being eaten on taste perception. To explore this, we focused on the Bouba-Kiki effect, illustrating an interaction between shape and several modalities, and investigated the effect of Bouba- and Kiki-shaped (rounded and angular) foods eaten on taste perception. We utilized a 3D food printer to produce four different shapes of chocolate pieces based on the Bouba-Kiki. Participants tasted each piece and completed a chocolate flavor questionnaire. With Bayesian analysis, we determined that the Bouba-shaped chocolate pieces were perceived as sweeter than the Kiki-shaped ones, supporting earlier studies on crossmodal correspondences between shape and taste perception. However, there were no significant differences in ratings of other tastes, such as sourness and bitterness. Our research indicates that shape can affect taste perception during consumption and suggests that 3D food printers offer an opportunity to design specific shapes that influence taste experiences.
Collapse
Affiliation(s)
- Kazuhiro Ogata
- Department of Design, Kyoto Institute of Technology, Kyoto, Japan
- OMRON SINIC X Corporation, Tokyo, Japan
| | - Reo Gakumi
- OMRON SINIC X Corporation, Tokyo, Japan
- Graduate School of Informatics, Kyoto University, Kyoto, Japan
| | | | | | | |
Collapse
|
5
|
Chen YC, Huang PC. Examining the automaticity and symmetry of sound-shape correspondences. Front Psychol 2023; 14:1172946. [PMID: 37342641 PMCID: PMC10277733 DOI: 10.3389/fpsyg.2023.1172946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 05/16/2023] [Indexed: 06/23/2023] Open
Abstract
Introduction A classic example of sound-shape correspondences is the mapping of the vowel /i/ with angular patterns and the vowel /u/ with rounded patterns. Such crossmodal correspondences have been reliably reported when tested in explicit matching tasks. Nevertheless, it remains unclear whether such sound-shape correspondences automatically occur and bidirectionally modulate people's perception. We address this question by adopting the explicit matching task and two implicit tasks. Methods In Experiment 1, we examined the sound-shape correspondences using the implicit association test (IAT), in which the sounds and shapes were both task-relevant, followed by an explicit matching task. In Experiments 2 and 3, we adopted the speeded classification task; when the target was a sound (or shape), a task-irrelevant shape (or sound) that was congruent or incongruent to the target was simultaneously presented. In addition, the participants performed the explicit matching task either before or after the speeded classification task. Results and Discussion The congruency effect was more pronounced in the IAT than in the speeded classification task; in addition, a bin analysis of RTs revealed that the congruency effect took time to develop. These findings suggest that the sound-shape correspondences were not completely automatic. The magnitude and onset of visual and auditory congruency effects were comparable, suggesting that the crossmodal modulations were symmetrical. Taken together, the sound-shape correspondences appeared not to be completely automatic, but their modulation was bidirectionally symmetrical once it occurred.
Collapse
Affiliation(s)
- Yi-Chuan Chen
- Department of Medicine, MacKay Medical College, New Taipei City, Taiwan
| | - Pi-Chun Huang
- Department of Psychology, National Cheng Kung University, Tainan, Taiwan
| |
Collapse
|
6
|
Kilpatrick A, Ćwiek A, Lewis E, Kawahara S. A cross-linguistic, sound symbolic relationship between labial consonants, voiced plosives, and Pokémon friendship. Front Psychol 2023; 14:1113143. [PMID: 36910799 PMCID: PMC10000297 DOI: 10.3389/fpsyg.2023.1113143] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 02/07/2023] [Indexed: 03/14/2023] Open
Abstract
Introduction This paper presents a cross-linguistic study of sound symbolism, analysing a six-language corpus of all Pokémon names available as of January 2022. It tests the effects of labial consonants and voiced plosives on a Pokémon attribute known as friendship. Friendship is a mechanic in the core series of Pokémon video games that arguably reflects how friendly each Pokémon is. Method Poisson regression is used to examine the relationship between the friendship mechanic and the number of times /p/, /b/, /d/, /m/, /g/, and /w/ occur in the names of English, Japanese, Korean, Chinese, German, and French Pokémon. Results Bilabial plosives, /p/ and /b/, typically represent high friendship values in Pokémon names while /m/, /d/, and /g/ typically represent low friendship values. No association is found for /w/ in any language. Discussion Many of the previously known cases of cross-linguistic sound symbolic patterns can be explained by the relationship between how sounds in words are articulated and the physical qualities of the referents. This study, however, builds upon the underexplored relationship between sound symbolism and abstract qualities.
Collapse
|
7
|
Zhang B. Sonja Taigel:
Ikonizität. ZEITSCHRIFT FÜR SPRACHWISSENSCHAFT 2022; 41:429-434. [DOI: 10.1515/zfs-2022-2003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
|
8
|
Resolving the bouba-kiki effect enigma by rooting iconic sound symbolism in physical properties of round and spiky objects. Sci Rep 2022; 12:19172. [PMID: 36357511 PMCID: PMC9649795 DOI: 10.1038/s41598-022-23623-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Accepted: 11/02/2022] [Indexed: 11/11/2022] Open
Abstract
The "bouba-kiki effect", where "bouba" is perceived round and "kiki" spiky, remains a puzzling enigma. We solve it by combining mathematical findings largely unknown in the field, with computational models and novel experimental evidence. We reveal that this effect relies on two acoustic cues: spectral balance and temporal continuity. We demonstrate that it is not speech-specific but rather rooted in physical properties of objects, creating audiovisual regularities in the environment. Round items are mathematically bound to produce, when hitting or rolling on a surface, lower-frequency spectra and more continuous sounds than same-size spiky objects. Finally, we show that adults are sensitive to such regularities. Hence, intuitive physics impacts language perception and possibly language acquisition and evolution too.
Collapse
|
9
|
Lee BP, Spence C. Crossmodal correspondences between basic tastes and visual design features: A narrative historical review. Iperception 2022; 13:20416695221127325. [PMID: 36246303 PMCID: PMC9558874 DOI: 10.1177/20416695221127325] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 09/01/2022] [Indexed: 11/26/2022] Open
Abstract
People tend to associate abstract visual features with basic taste qualities. This narrative historical review critically evaluates the literature on these associations, often referred to as crossmodal correspondences, between basic tastes and visual design features such as color hue and shape curvilinearity. The patterns, discrepancies, and evolution in the development of the research are highlighted while the mappings that have been reported to date are summarized. The review also reflects on issues of cross-cultural validity and deviations in the matching patterns that are observed when correspondences are assessed with actual tastants versus with verbal stimuli. The various theories that have been proposed to account for different classes of crossmodal correspondence are discussed, among which the statistical and affective (or emotional-mediation) accounts currently appear most promising. Several critical research questions for the future are presented to address the gaps that have been identified in the literature and help validate the popular theories on the origin and operations of visual-taste correspondences.
Collapse
Affiliation(s)
- Byron P. Lee
- Byron P. Lee, New Radcliffe House,
Radcliffe Observatory Quarter, Woodstock Road, Oxford OX2 6GG, UK.
| | | |
Collapse
|
10
|
de Varda AG, Strapparava C. A Cross-Modal and Cross-lingual Study of Iconicity in Language: Insights From Deep Learning. Cogn Sci 2022; 46:e13147. [PMID: 35665953 PMCID: PMC9285447 DOI: 10.1111/cogs.13147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 04/27/2022] [Accepted: 04/28/2022] [Indexed: 11/30/2022]
Abstract
The present paper addresses the study of non-arbitrariness in language within a deep learning framework. We present a set of experiments aimed at assessing the pervasiveness of different forms of non-arbitrary phonological patterns across a set of typologically distant languages. Different sequence-processing neural networks are trained in a set of languages to associate the phonetic vectorization of a set of words to their sensory (Experiment 1), semantic (Experiment 2), and word-class representations (Experiment 3). The models are then tested, without further training, in a set of novel instances in a language belonging to a different language family, and their performance is compared with a randomized baseline. We show that the three cross-domain mappings can be successfully transferred across languages and language families, suggesting that the phonological structure of the lexicon is pervaded with language-invariant cues about the words' meaning and their syntactic classes.
Collapse
|
11
|
Ćwiek A, Fuchs S, Draxler C, Asu EL, Dediu D, Hiovain K, Kawahara S, Koutalidis S, Krifka M, Lippus P, Lupyan G, Oh GE, Paul J, Petrone C, Ridouane R, Reiter S, Schümchen N, Szalontai Á, Ünal-Logacev Ö, Zeller J, Perlman M, Winter B. The bouba/kiki effect is robust across cultures and writing systems. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200390. [PMID: 34775818 PMCID: PMC8591387 DOI: 10.1098/rstb.2020.0390] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Accepted: 09/10/2021] [Indexed: 01/06/2023] Open
Abstract
The bouba/kiki effect-the association of the nonce word bouba with a round shape and kiki with a spiky shape-is a type of correspondence between speech sounds and visual properties with potentially deep implications for the evolution of spoken language. However, there is debate over the robustness of the effect across cultures and the influence of orthography. We report an online experiment that tested the bouba/kiki effect across speakers of 25 languages representing nine language families and 10 writing systems. Overall, we found strong evidence for the effect across languages, with bouba eliciting more congruent responses than kiki. Participants who spoke languages with Roman scripts were only marginally more likely to show the effect, and analysis of the orthographic shape of the words in different scripts showed that the effect was no stronger for scripts that use rounder forms for bouba and spikier forms for kiki. These results confirm that the bouba/kiki phenomenon is rooted in crossmodal correspondence between aspects of the voice and visual shape, largely independent of orthography. They provide the strongest demonstration to date that the bouba/kiki effect is robust across cultures and writing systems. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.
Collapse
Affiliation(s)
- Aleksandra Ćwiek
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
- Institut für deutsche Sprache und Linguistik, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
| | - Susanne Fuchs
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
| | - Christoph Draxler
- Institute of Phonetics and Speech Processing, Ludwig Maximilian University, 80799 Munich, Germany
| | - Eva Liina Asu
- Institute of Estonian and General Linguistics, University of Tartu, 50090 Tartu, Estonia
| | - Dan Dediu
- Laboratoire Dynamique Du Langage UMR 5596, Université Lumière Lyon 2, 69363 Lyon, France
| | - Katri Hiovain
- Department of Digital Humanities, University of Helsinki, 00014 Helsinki, Finland
| | - Shigeto Kawahara
- The Institute of Cultural and Linguistic Studies, Keio University, Mita Minatoku, Tokyo 108-8345, Japan
| | - Sofia Koutalidis
- Faculty of Linguistics and Literary Studies, Bielefeld University, 33615 Bielefeld, Germany
| | - Manfred Krifka
- Leibniz-Zentrum Allgemeine Sprachwissenschaft, 10117 Berlin, Germany
- Institut für deutsche Sprache und Linguistik, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
| | - Pärtel Lippus
- Institute of Estonian and General Linguistics, University of Tartu, 50090 Tartu, Estonia
| | - Gary Lupyan
- Department of Psychology, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Grace E. Oh
- Department of English Language and Literature, Konkuk University, Seoul 05029, South Korea
| | - Jing Paul
- Asian Studies Program, Agnes Scott College, Decatur, GA 30030, USA
| | - Caterina Petrone
- Aix-Marseille Université, CNRS, Laboratoire Parole et Langage, UMR 7309, 13100 Aix-en-Provence, France
| | - Rachid Ridouane
- Laboratoire de Phonétique et Phonologie, UMR 7018, CNRS and Sorbonne Nouvelle, 75005 Paris, France
| | - Sabine Reiter
- Depto. de Polonês, Alemão e Letras Clássicas, Universidade Federal do Paraná, 80060-150 Curitiba, Brazil
| | - Nathalie Schümchen
- Department of Language and Communication, University of Southern Denmark, 5230 Odense, Denmark
| | - Ádám Szalontai
- Department of Phonetics, Hungarian Research Centre for Linguistics, Budapest 1068, Hungary
| | - Özlem Ünal-Logacev
- School of Health Sciences, Department of Speech and Language Therapy, Istanbul Medipol University, 34810 Istanbul, Turkey
| | - Jochen Zeller
- School of Arts, Linguistics Discipline, University of KwaZulu-Natal, Durban 4041, South Africa
| | - Marcus Perlman
- Department of English Language and Linguistics, University of Birmingham, Birmingham B15 2TT, UK
| | - Bodo Winter
- Department of English Language and Linguistics, University of Birmingham, Birmingham B15 2TT, UK
| |
Collapse
|
12
|
Chow HM, Harris DA, Eid S, Ciaramitaro VM. The feeling of "kiki": Comparing developmental changes in sound-shape correspondence for audio-visual and audio-tactile stimuli. J Exp Child Psychol 2021; 209:105167. [PMID: 33915481 DOI: 10.1016/j.jecp.2021.105167] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2020] [Revised: 03/21/2021] [Accepted: 03/22/2021] [Indexed: 10/21/2022]
Abstract
Sound-shape crossmodal correspondence, the naturally occurring associations between abstract visual shapes and nonsense sounds, is one aspect of multisensory processing that strengthens across early childhood. Little is known regarding whether school-aged children exhibit other variants of sound-shape correspondences such as audio-tactile (AT) associations between tactile shapes and nonsense sounds. Based on previous research in blind individuals suggesting the role of visual experience in establishing sound-shape correspondence, we hypothesized that children would show weaker AT association than adults and that children's AT association would be enhanced with visual experience of the shapes. In Experiment 1, we showed that, when asked to match shapes explored haptically via touch to nonsense words, 6- to 8-year-olds exhibited inconsistent AT associations, whereas older children and adults exhibited the expected AT associations, despite robust audio-visual (AV) associations found across all age groups in a related study. In Experiment 2, we confirmed the role of visual experience in enhancing AT association; here, 6- to 8-year-olds could exhibit the expected AT association if first exposed to the AV condition, whereas adults showed the expected AT association irrespective of whether the AV condition was tested first or second. Our finding suggests that AT sound-shape correspondence is weak early in development relative to AV sound-shape correspondence, paralleling previous findings on the development of other types of multisensory associations. The potential role of visual experience in the development of sound-shape correspondences in other senses is discussed.
Collapse
Affiliation(s)
- Hiu Mei Chow
- Department of Psychology, Developmental and Brain Sciences, University of Massachusetts Boston, Boston, MA 02125, USA; Department of Ophthalmology and Visual Sciences, University of British Columbia, Vancouver, British Columbia V6T 1Z4, Canada
| | - Daniel A Harris
- Department of Psychology, Developmental and Brain Sciences, University of Massachusetts Boston, Boston, MA 02125, USA; Division of Epidemiology, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario M5T 3M7, Canada
| | - Sandy Eid
- Department of Psychology, Developmental and Brain Sciences, University of Massachusetts Boston, Boston, MA 02125, USA
| | - Vivian M Ciaramitaro
- Department of Psychology, Developmental and Brain Sciences, University of Massachusetts Boston, Boston, MA 02125, USA.
| |
Collapse
|
13
|
Lacey S, Jamal Y, List SM, McCormick K, Sathian K, Nygaard LC. Stimulus Parameters Underlying Sound-Symbolic Mapping of Auditory Pseudowords to Visual Shapes. Cogn Sci 2020; 44:e12883. [PMID: 32909637 PMCID: PMC7896554 DOI: 10.1111/cogs.12883] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2018] [Revised: 06/06/2020] [Accepted: 07/01/2020] [Indexed: 12/12/2022]
Abstract
Sound symbolism refers to non-arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as "maluma" and "takete" with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics-to-noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound-to-shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross-modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.
Collapse
Affiliation(s)
- Simon Lacey
- Department of Neurology, Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, PA 17033-0859, USA
- Department of Neural & Behavioral Sciences, Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, PA 17033-0859, USA
- Department of Neurology, Emory University, Atlanta, GA 30322, USA
| | - Yaseen Jamal
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| | - Sara M. List
- Department of Neurology, Emory University, Atlanta, GA 30322, USA
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| | - Kelly McCormick
- Department of Neurology, Emory University, Atlanta, GA 30322, USA
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| | - K. Sathian
- Department of Neurology, Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, PA 17033-0859, USA
- Department of Neural & Behavioral Sciences, Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, PA 17033-0859, USA
- Department of Psychology, Milton S. Hershey Medical Center, Penn State College of Medicine, Hershey, PA 17033-0859, USA
- Department of Neurology, Emory University, Atlanta, GA 30322, USA
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| | - Lynne C. Nygaard
- Department of Psychology, Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
14
|
Sourav S, Kekunnaya R, Shareef I, Banerjee S, Bottari D, Röder B. A Protracted Sensitive Period Regulates the Development of Cross-Modal Sound-Shape Associations in Humans. Psychol Sci 2019; 30:1473-1482. [PMID: 31483197 PMCID: PMC6787766 DOI: 10.1177/0956797619866625] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Humans preferentially match arbitrary words containing higher- and lower-frequency phonemes to angular and smooth shapes, respectively. Here, we investigated the role of visual experience in the development of audiovisual and audiohaptic sound-shape associations (SSAs) using a unique set of five groups: individuals who had suffered a transient period of congenital blindness through congenital bilateral dense cataracts before undergoing cataract-reversal surgeries (CC group), individuals with a history of developmental cataracts (DC group), individuals with congenital permanent blindness (CB group), individuals with late permanent blindness (LB group), and controls with typical sight (TS group). Whereas the TS and LB groups showed highly robust SSAs, the CB, CC, and DC groups did not-in any of the modality combinations tested. These results provide evidence for a protracted sensitive period during which aberrant vision prevents SSA acquisition. Moreover, the finding of a systematic SSA in the LB group demonstrates that representations acquired during the sensitive period are resilient to loss despite dramatically changed experience.
Collapse
Affiliation(s)
- Suddha Sourav
- Biological Psychology and Neuropsychology, University of Hamburg
| | - Ramesh Kekunnaya
- Child Sight Institute, Jasti V. Ramanamma Children's Eye Care Center, LV Prasad Eye Institute, Hyderabad, India
| | - Idris Shareef
- Child Sight Institute, Jasti V. Ramanamma Children's Eye Care Center, LV Prasad Eye Institute, Hyderabad, India
| | - Seema Banerjee
- Child Sight Institute, Jasti V. Ramanamma Children's Eye Care Center, LV Prasad Eye Institute, Hyderabad, India.,School of Optometry, The Hong Kong Polytechnic University
| | - Davide Bottari
- Biological Psychology and Neuropsychology, University of Hamburg.,IMT School for Advanced Studies Lucca
| | - Brigitte Röder
- Biological Psychology and Neuropsychology, University of Hamburg
| |
Collapse
|
15
|
Abstract
The present study aimed to investigate whether or not the so-called "bouba-kiki" effect is mediated by speech-specific representations. Sine-wave versions of naturally produced pseudowords were used as auditory stimuli in an implicit association task (IAT) and an explicit cross-modal matching (CMM) task to examine cross-modal shape-sound correspondences. A group of participants trained to hear the sine-wave stimuli as speech was compared to a group that heard them as non-speech sounds. Sound-shape correspondence effects were observed in both groups and tasks, indicating that speech-specific processing is not fundamental to the "bouba-kiki" phenomenon. Effects were similar across groups in the IAT, while in the CMM task the speech-mode group showed a stronger effect compared with the non-speech group. This indicates that, while both tasks reflect auditory-visual associations, only the CMM task is additionally sensitive to associations involving speech-specific representations.
Collapse
|
16
|
I know that "Kiki" is angular: The metacognition underlying sound-shape correspondences. Psychon Bull Rev 2019; 26:261-268. [PMID: 30097975 DOI: 10.3758/s13423-018-1516-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
We examined the ability of people to evaluate their confidence when making perceptual judgments concerning a classic crossmodal correspondence, the Bouba/Kiki effect: People typically match the "Bouba" sound to more rounded patterns and match the "Kiki" sound to more angular patterns instead. For each visual pattern, individual participants were more confident about their own matching judgments when they happened to fall in line with the consensual response regarding whether the pattern was rated as "Bouba" or "Kiki". Logit regression analyses demonstrated that participants' confidence ratings and matching judgments were predictable by similar regression functions. This implies that the consensus and confidence underlying the Bouba/Kiki effect are underpinned by a common process, whereby perceptual features in the patterns are extracted and then used to match the sound according to rules of crossmodal correspondences. Combining both matching and confidence measures potentially allows one to explore and quantify the strength of associations in human knowledge.
Collapse
|
17
|
Dreksler N, Spence C. A Critical Analysis of Colour-Shape Correspondences: Examining the Replicability of Colour-Shape Associations. Iperception 2019; 10:2041669519834042. [PMID: 30956786 PMCID: PMC6442080 DOI: 10.1177/2041669519834042] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2018] [Accepted: 02/05/2019] [Indexed: 12/04/2022] Open
Abstract
Research on the topic of colour-shape correspondences started in the early 20th century with the Bauhaus artist Wassily Kandinsky. However, more recently, the topic has been examined using the empirical framework of crossmodal correspondences research. The field remains one in which consistent results and generalisable hypotheses about the existence and nature of colour-shape correspondences are lacking. The replicability and consistency of findings concerning colour-shape correspondences are examined in three online colour-shape matching experiments using the same procedure and study design while varying the sets of shape stimuli that are evaluated. Participants matched one of 36 colours to each shape as well as made preference and arousal appraisal ratings for each of the shapes and colours. The complexities of analysing colour-shape correspondence data are discussed and illustrated by classifying and analysing shape and colours in a variety of different ways, including using continuous perceptual and objective measures. Significant colour-shape associations were found. However, as hypothesised, limited consistent results in regard to what perceptual shape characteristics predicted colour choices were documented across the three stimuli sets. This was the case both within and across different analysis methods. The factors that may be responsible for these inconsistencies are critically discussed. Intriguingly, however, evidence for emotional mediation, whereby shape and colour liking and arousal appraisals appear to influence the colour-shape correspondences made by participants, was found across all three experiments.
Collapse
Affiliation(s)
- Noemi Dreksler
- Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, UK
| | - Charles Spence
- Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, UK
| |
Collapse
|
18
|
Peiffer-Smadja N, Cohen L. The cerebral bases of the bouba-kiki effect. Neuroimage 2019; 186:679-689. [PMID: 30503933 DOI: 10.1016/j.neuroimage.2018.11.033] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2018] [Revised: 11/18/2018] [Accepted: 11/21/2018] [Indexed: 11/30/2022] Open
Abstract
The crossmodal correspondence between some speech sounds and some geometrical shapes, known as the bouba-kiki (BK) effect, constitutes a remarkable exception to the general arbitrariness of the links between word meaning and word sounds. We have analyzed the association of shapes and sounds in order to determine whether it occurs at a perceptual or at a decisional level, and whether it takes place in sensory cortices or in supramodal regions. First, using an Implicit Association Test (IAT), we have shown that the BK effect may occur without participants making any explicit decision relative to sound-shape associations. Second, looking for the brain correlates of implicit BK matching, we have found that intermodal matching influences activations in both auditory and visual sensory cortices. Moreover, we found stronger prefrontal activation to mismatching than to matching stimuli, presumably reflecting a modulation of executive processes by crossmodal correspondence. Thus, through its roots in the physiology of object categorization and crossmodal matching, the BK effect provides a unique insight into some non-linguistic components of word formation.
Collapse
Affiliation(s)
- Nathan Peiffer-Smadja
- Institut du Cerveau et de la Moelle épinière, ICM, Inserm U 1127, CNRS UMR 7225, Sorbonne Université, F-75013, Paris, France
| | - Laurent Cohen
- Institut du Cerveau et de la Moelle épinière, ICM, Inserm U 1127, CNRS UMR 7225, Sorbonne Université, F-75013, Paris, France; Département de Neurologie 1, Hôpital de la Pitié Salpêtrière, AP-HP, F-75013, Paris, France.
| |
Collapse
|
19
|
Sound symbolism in sighted and blind. The role of vision and orthography in sound-shape correspondences. Cognition 2019; 185:62-70. [PMID: 30660923 DOI: 10.1016/j.cognition.2019.01.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2017] [Revised: 08/28/2018] [Accepted: 01/07/2019] [Indexed: 10/27/2022]
Abstract
Non-arbitrary sound-shape correspondences (SSC), such as the "bouba-kiki" effect, have been consistently observed across languages and together with other sound-symbolic phenomena challenge the classic linguistic dictum of the arbitrariness of the sign. Yet, it is unclear what makes a sound "round" or "spiky" to the human mind. Here we tested the hypothesis that visual experience is necessary for the emergence of SSC, supported by empirical evidence showing reduced SSC in visually impaired people. Results of two experiments comparing early blind and sighted individuals showed that SSC emerged strongly in both groups. Experiment 2, however, showed a partially different pattern of SSC in sighted and blind, that was mostly explained by a different effect of orthographic letter shape: The shape of written letters (spontaneously activated by spoken words) influenced SSC in the sighted, but not in the blind, who are exposed to an orthography (Braille) in which letters do not have spiky or round outlines. In sum, early blindness does not prevent the emergence of SSC, and differences between sighted and visually impaired people may be due the indirect influence (or lack thereof) of orthographic letter shape.
Collapse
|
20
|
De Carolis L, Marsico E, Arnaud V, Coupé C. Assessing sound symbolism: Investigating phonetic forms, visual shapes and letter fonts in an implicit bouba-kiki experimental paradigm. PLoS One 2018; 13:e0208874. [PMID: 30576331 PMCID: PMC6303039 DOI: 10.1371/journal.pone.0208874] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2017] [Accepted: 11/27/2018] [Indexed: 11/19/2022] Open
Abstract
Classically, in the bouba-kiki association task, a subject is asked to find the best association between one of two shapes–a round one and a spiky one–and one of two pseudowords–bouba and kiki. Numerous studies report that spiky shapes are associated with kiki, and round shapes with bouba. This task is likely the most prevalent in the study of non-conventional relationships between linguistic forms and meanings, also known as sound symbolism. However, associative tasks are explicit in the sense that they highlight phonetic and visual contrasts and require subjects to establish a crossmodal link between stimuli of different natures. Additionally, recent studies have raised the question whether visual resemblances between the target shapes and the letters explain the pattern of association, at least in literate subjects. In this paper, we report a more implicit testing paradigm of the bouba-kiki effect with the use of a lexical decision task with character strings presented in round or spiky frames. Pseudowords and words are, furthermore, displayed with either an angular or a curvy font to investigate possible graphemic bias. Innovative analyses of response times are performed with GAMLSS models, which offer a large range of possible distributions of error terms, and a generalized Gama distribution is found to be the most appropriate. No sound symbolic effect appears to be significant, but an interaction effect is in particular observed between spiky shapes and angular letters leading to faster response times. We discuss these results with respect to the visual saliency of angular shapes, priming, brain activation, synaesthesia and ideasthesia.
Collapse
Affiliation(s)
- Léa De Carolis
- Laboratoire Dynamique du Langage, CNRS & Université de Lyon, Lyon, France
| | - Egidio Marsico
- Laboratoire Dynamique du Langage, CNRS & Université de Lyon, Lyon, France
| | - Vincent Arnaud
- Département des arts et lettres, Université du Québec à Chicoutimi, Chicoutimi, Canada
| | - Christophe Coupé
- Laboratoire Dynamique du Langage, CNRS & Université de Lyon, Lyon, France
- Department of Linguistics, The University of Hong Kong, Hong Kong SAR, China
- * E-mail:
| |
Collapse
|
21
|
Sakamoto M, Watanabe J. Bouba/Kiki in Touch: Associations Between Tactile Perceptual Qualities and Japanese Phonemes. Front Psychol 2018; 9:295. [PMID: 29593602 PMCID: PMC5857596 DOI: 10.3389/fpsyg.2018.00295] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2017] [Accepted: 02/22/2018] [Indexed: 11/28/2022] Open
Abstract
Several studies have shown cross-modal associations between sounds and vision or gustation by asking participants to match pre-defined sound-symbolic words (SSWs), such as “bouba” or “kiki,” with visual or gustatory materials. Here, we conducted an explorative study on cross-modal associations of tactile sensations using spontaneous production of Japanese SSWs and semantic ratings. The Japanese language was selected, because it has a large number of SSWs that can represent a wide range of tactile perceptual spaces with fine resolution, and it shows strong associations between sound and touch. In the experiment, we used 120 everyday materials to cover basic material categories that could be associated with fundamental dimensions of tactile perception. Upon contact with these materials, participants expressed their tactile sensations by using Japanese SSWs, and at the same time, evaluated the tactile sensations by semantic differential scales using adjective pairs. Thanks to the variety of testing materials, we were able to demonstrate the existence of systematic associations between sounds and tactile fundamental perceptual dimensions in a more detailed and comprehensive way than ever done so before. In particular, we found that for vowels, positive tactile ratings were associated with the back vowel (/u/), while negative ratings were associated with the front vowels (/i/ and /e/). The central vowels (/o/ and /a/) were mainly associated with rough, hard, and dry feelings. Consonants were categorized based on vocal features and articulation. The category of the voiced consonants (e.g., /dz/ and /g/) corresponded to feelings of roughness, while that of voiceless consonants (e.g., /ʦ/, and /s/) corresponded to feelings of smoothness. The categories of the bilabial plosive (/p/ and /b/) and voiced alveolar nasal (/n/) consonants were mainly related to soft, sticky and wet feelings, while that of voiceless alveolar affricate (/ʦ/) and voiceless velar plosive (/k/) consonants were related to hard, slippery and dry feelings.
Collapse
Affiliation(s)
- Maki Sakamoto
- Department of Informatics, The University of Electro-Communications, Tokyo, Japan
| | - Junji Watanabe
- NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Kanagawa, Japan
| |
Collapse
|
22
|
Shang N, Styles SJ. Is a High Tone Pointy? Speakers of Different Languages Match Mandarin Chinese Tones to Visual Shapes Differently. Front Psychol 2017; 8:2139. [PMID: 29270147 PMCID: PMC5726031 DOI: 10.3389/fpsyg.2017.02139] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2017] [Accepted: 11/24/2017] [Indexed: 11/13/2022] Open
Abstract
Studies investigating cross-modal correspondences between auditory pitch and visual shapes have shown children and adults consistently match high pitch to pointy shapes and low pitch to curvy shapes, yet no studies have investigated linguistic-uses of pitch. In the present study, we used a bouba/kiki style task to investigate the sound/shape mappings for Tones of Mandarin Chinese, for three groups of participants with different language backgrounds. We recorded the vowels [i] and [u] articulated in each of the four tones of Mandarin Chinese. In Study 1 a single auditory stimulus was presented with two images (one curvy, one spiky). In Study 2 a single image was presented with two auditory stimuli differing only in tone. Participants were asked to select the best match in an online ‘Quiz.’ Across both studies, we replicated the previously observed ‘u-curvy, i-pointy’ sound/shape cross-modal correspondence in all groups. However, Tones were mapped differently by people with different language backgrounds: speakers of Mandarin Chinese classified as Chinese-dominant systematically matched Tone 1 (high, steady) to the curvy shape and Tone 4 (falling) to the pointy shape, while English speakers with no knowledge of Chinese preferred to match Tone 1 (high, steady) to the pointy shape and Tone 3 (low, dipping) to the curvy shape. These effects were observed most clearly in Study 2 where tone-pairs were contrasted explicitly. These findings are in line with the dominant patterns of linguistic pitch perception for speakers of these languages (pitch-change, and pitch height, respectively). Chinese English balanced bilinguals showed a bivalent pattern, swapping between the Chinese pitch-change pattern and the English pitch-height pattern depending on the task. These findings show for that the supposedly universal pattern of mapping linguistic sounds to shape is modulated by the sensory properties of a speaker’s language system, and that people with high functioning in more than one language can dynamically shift between patterns.
Collapse
Affiliation(s)
- Nan Shang
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| | - Suzy J Styles
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
23
|
Symmetry and its role in the crossmodal correspondence between shape and taste. Atten Percept Psychophys 2017; 80:738-751. [DOI: 10.3758/s13414-017-1463-x] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
24
|
Turoman N, Styles SJ. Glyph guessing for 'oo' and 'ee': spatial frequency information in sound symbolic matching for ancient and unfamiliar scripts. ROYAL SOCIETY OPEN SCIENCE 2017; 4:170882. [PMID: 28989784 PMCID: PMC5627124 DOI: 10.1098/rsos.170882] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2017] [Accepted: 08/14/2017] [Indexed: 06/07/2023]
Abstract
In three experiments, we asked whether diverse scripts contain interpretable information about the speech sounds they represent. When presented with a pair of unfamiliar letters, adult readers correctly guess which is /i/ (the 'ee' sound in 'feet'), and which is /u/ (the 'oo' sound in 'shoe') at rates higher than expected by chance, as shown in a large sample of Singaporean university students (Experiment 1) and replicated in a larger sample of international Internet users (Experiment 2). To uncover what properties of the letters contribute to different scripts' 'guessability,' we analysed the visual spatial frequencies in each letter (Experiment 3). We predicted that the lower spectral frequencies in the formants of the vowel /u/ would pattern with lower spatial frequencies in the corresponding letters. Instead, we found that across all spatial frequencies, the letter with more black/white cycles (i.e. more ink) was more likely to be guessed as /u/, and the larger the difference between the glyphs in a pair, the higher the script's guessability. We propose that diverse groups of humans across historical time and geographical space tend to employ similar iconic strategies for representing speech in visual form, and provide norms for letter pairs from 56 diverse scripts.
Collapse
Affiliation(s)
- Nora Turoman
- Department of Radiology, Centre Hospitalier Universitaire Vaudois (CHUV) and University of Lausanne (UniL), Rue du Bugnon 46, 1011 Lausanne, Switzerland
- Department of Clinical Neurosciences, Centre Hospitalier Universitaire Vaudois (CHUV) and University of Lausanne (UniL), Rue du Bugnon 46, 1011 Lausanne, Switzerland
| | - Suzy J. Styles
- Division of Psychology, School of Humanities and Social Sciences, Nanyang Technological University, 14 Nanyang Drive, Singapore
| |
Collapse
|
25
|
Chen YC, Spence C. Assessing the Role of the 'Unity Assumption' on Multisensory Integration: A Review. Front Psychol 2017; 8:445. [PMID: 28408890 PMCID: PMC5374162 DOI: 10.3389/fpsyg.2017.00445] [Citation(s) in RCA: 79] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2016] [Accepted: 03/09/2017] [Indexed: 01/20/2023] Open
Abstract
There has been longstanding interest from both experimental psychologists and cognitive neuroscientists in the potential modulatory role of various top-down factors on multisensory integration/perception in humans. One such top-down influence, often referred to in the literature as the 'unity assumption,' is thought to occur in those situations in which an observer considers that various of the unisensory stimuli that they have been presented with belong to one and the same object or event (Welch and Warren, 1980). Here, we review the possible factors that may lead to the emergence of the unity assumption. We then critically evaluate the evidence concerning the consequences of the unity assumption from studies of the spatial and temporal ventriloquism effects, from the McGurk effect, and from the Colavita visual dominance paradigm. The research that has been published to date using these tasks provides support for the claim that the unity assumption influences multisensory perception under at least a subset of experimental conditions. We then consider whether the notion has been superseded in recent years by the introduction of priors in Bayesian causal inference models of human multisensory perception. We suggest that the prior of common cause (that is, the prior concerning whether multisensory signals originate from the same source or not) offers the most useful way to quantify the unity assumption as a continuous cognitive variable.
Collapse
Affiliation(s)
| | - Charles Spence
- Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford UniversityOxford, UK
| |
Collapse
|