Cross-modal association between vowels and colours: A cross-linguistic perspective

Iconicity mediates semantic networks of sound symbolisma)

One speech sound can be associated with multiple meanings through iconicity, indexicality, and/or systematicity. It was not until recently that this "pluripotentiality" of sound symbolism attracted serious attention, and it remains uninvestigated how pluripotentiality may arise. In the current study, Japanese, Korean, Mandarin, and English speakers rated unfamiliar jewel names on three semantic scales: size, brightness, and hardness. The results showed language-specific and cross-linguistically shared pluripotential sound symbolism. Japanese speakers associated voiced stops with large and dark jewels, whereas Mandarin speakers associated [i] with small and bright jewels. Japanese, Mandarin, and English speakers also associated lip rounding with darkness and softness. These sound-symbolic meanings are unlikely to be obtained through metaphorical or metonymical extension, nor are they reported to colexify. Notably, in a purely semantic network without the mediation of lip rounding, softness can instead be associated with brightness, as illustrated by synesthetic metaphors such as yawaraka-na hizashi /jawaɾakanaçizaɕi/ "a gentle (lit. soft) sunshine" in Japanese. These findings suggest that the semantic networks of sound symbolism may not coincide with those of metaphor or metonymy. The current study summarizes the findings in the form of (phono)semantic maps to facilitate cross-linguistic comparisons of pluripotential sound symbolism.

Cross-modal associations between paintings and sounds: Effects of embodiment

The present study investigated cross-modal associations between a series of paintings and sounds. We studied the effects of sound congruency (congruent vs. non-congruent sounds) and embodiment (embodied vs. synthetic sounds) on the evaluation of abstract and figurative paintings. Participants evaluated figurative and abstract paintings paired with congruent and non-congruent embodied and synthetic sounds. They also evaluated the perceived meaningfulness of the paintings, aesthetic value and immersive experience of the paintings. Embodied sounds (sounds associated with bodily sensations, bodily movements and touch) were more strongly associated with figurative paintings, while synthetic sounds (non-embodied sounds) were more strongly associated with abstract paintings. Sound congruency increased the perceived meaningfulness, immersive experience and aesthetic value of paintings. Sound embodiment increased immersive experience of paintings.

Distinct colours in the 'synaesthetic colour palette'

In grapheme-colour synaesthesia, particular linguistic elements evoke particular colour sensations. Interestingly, when asked, non-synaesthetes can also associate colours to letters, and previous studies show that specific letter-to-colour associations have similar biases to those of synaesthetes. However, it is an open question whether the colours reported by synaesthetes and non-synaesthetes differ overall: is there a 'synaesthetic colour palette'? In this study, we visualize the overall distribution in colour space of colour concurrents in grapheme-colour synaesthetes, and colour associations in non-synaesthetic controls. We confirm the existence of a synaesthetic colour palette: colour concurrents in synaesthetes are different from colour associations in non-synaesthetes. We quantify three factors that distinguish the colour palette of synaesthetes and non-synaesthetes: synaesthetes have an increased over-representation of 'pure' (unmixed) hues, an increased presence of 'warm' (yellow, orange, brown) colours, and an increased presence of achromatic (grey, white, black) colours. Furthermore, we demonstrate that differences in the synaesthetic colour palette can be used to train a machine learning algorithm to reliably classify single subjects as synaesthetes versus non-synaesthetes without using test-retest consistency data. As far as we know, this is the first time an individual could be 'diagnosed' as a synaesthete, based only on his or her colours evoked by letters. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.