What makes a shape “baba”? The shape features prioritized in sound–shape correspondence change with development

Prior visual experience increases children's use of effective haptic exploration strategies in audio-tactile sound-shape correspondences

Sound-shape correspondence refers to the preferential mapping of information across the senses, such as associating a nonsense word like bouba with rounded abstract shapes and kiki with spiky abstract shapes. Here we focused on audio-tactile (AT) sound-shape correspondences between nonsense words and abstract shapes that are felt but not seen. Despite previous research indicating a role for visual experience in establishing AT associations, it remains unclear how visual experience facilitates AT correspondences. Here we investigated one hypothesis: seeing the abstract shapes improve haptic exploration by (a) increasing effective haptic strategies and/or (b) decreasing ineffective haptic strategies. We analyzed five haptic strategies in video-recordings of 6- to 8-year-old children obtained in a previous study. We found the dominant strategy used to explore shapes differed based on visual experience. Effective strategies, which provide information about shape, were dominant in participants with prior visual experience, whereas ineffective strategies, which do not provide information about shape, were dominant in participants without prior visual experience. With prior visual experience, poking-an effective and efficient strategy-was dominant, whereas without prior visual experience, uncategorizable and ineffective strategies were dominant. These findings suggest that prior visual experience of abstract shapes in 6- to 8-year-olds can increase the effectiveness and efficiency of haptic exploration, potentially explaining why prior visual experience can increase the strength of AT sound-shape correspondences.

Crossmodal correspondences between visual features and tastes in preschoolers: an exploratory study

Introduction

Adults possess a natural inclination to associate sensory cues derived from distinct modalities, such as the pairing of sweet with pink. However, studies exploring crossmodal correspondences in children, particularly in the sensory pairing of visual features and tastes, are scant, leaving unanswered questions regarding the developmental trajectory of crossmodal correspondences. The present study investigates whether Japanese preschool children demonstrate specific biases in shape-color, shape-taste, and color-taste associations.

Methods

In a series of in-person experiments, 92 children between 3 to 6 years of age completed matching tasks utilizing paper stimuli.

Results

Children exhibit crossmodal correspondences in shape-color (circle-red and asymmetrical star-yellow), shape-taste (triangle-salty and circle-sweet), and color-taste (yellow-sour, black-bitter, and pink-sweet) associations. Moreover, children's choices are not influenced by their individual preferences.

Discussion

The crossmodal correspondences observed in this study have been observed in previous research on adults from the same (Japanese) culture, although adults showed more crossmodal correspondences than the children in this study (e.g., pink-circle, triangle-sour, and green-bitter). Thus, while some crossmodal correspondences emerge during childhood, others may require additional time to develop, thereby highlighting the importance of understanding the cognitive mechanisms underlying crossmodal correspondences from an ontogenic perspective.

The feeling of "kiki": Comparing developmental changes in sound-shape correspondence for audio-visual and audio-tactile stimuli

Sound-shape crossmodal correspondence, the naturally occurring associations between abstract visual shapes and nonsense sounds, is one aspect of multisensory processing that strengthens across early childhood. Little is known regarding whether school-aged children exhibit other variants of sound-shape correspondences such as audio-tactile (AT) associations between tactile shapes and nonsense sounds. Based on previous research in blind individuals suggesting the role of visual experience in establishing sound-shape correspondence, we hypothesized that children would show weaker AT association than adults and that children's AT association would be enhanced with visual experience of the shapes. In Experiment 1, we showed that, when asked to match shapes explored haptically via touch to nonsense words, 6- to 8-year-olds exhibited inconsistent AT associations, whereas older children and adults exhibited the expected AT associations, despite robust audio-visual (AV) associations found across all age groups in a related study. In Experiment 2, we confirmed the role of visual experience in enhancing AT association; here, 6- to 8-year-olds could exhibit the expected AT association if first exposed to the AV condition, whereas adults showed the expected AT association irrespective of whether the AV condition was tested first or second. Our finding suggests that AT sound-shape correspondence is weak early in development relative to AV sound-shape correspondence, paralleling previous findings on the development of other types of multisensory associations. The potential role of visual experience in the development of sound-shape correspondences in other senses is discussed.

Action sound-shape congruencies explain sound symbolism

Sound symbolism, the surprising semantic relationship between meaningless pseudowords (e.g., 'maluma', 'takete') and abstract (round vs. sharp) shapes, is a hitherto unexplained human-specific knowledge domain. Here we explore whether abstract sound symbolic links can be explained by those between the sounds and shapes of bodily actions. To this end, we asked human subjects to match pseudowords with abstract shapes and, in a different experimental block, the sounds of actions with the shapes of the trajectories of the actions causing these same sounds. Crucially, both conditions were also crossed. Our findings reveal concordant matching in the sound symbolic and action domains, and, importantly, significant correlations between them. We conclude that the sound symbolic knowledge interlinking speech sounds and abstract shapes is explained by audiovisual information immanent to action experience along with acoustic similarities between speech and action sounds. These results demonstrate a fundamental role of action knowledge for abstract sound symbolism, which may have been key to human symbol-manipulation ability.

Bouba and Kiki inside objects: Sound-shape correspondence for objects with a hole

Visual holes (cutouts in a surface) have recently intrigued vision scientists as interesting and useful stimuli in the studies of shape perception and as a perceptual conundrum regarding figure/ground organization. Adopting the Bouba/Kiki paradigm, this study addressed a controversial issue of whether the perceived shape of a closed region alters when the region changes from a solid object to an empty hole, in a more direct manner than previous studies did. Observers were presented with two doughnut-like cardboard cutouts, one with a flower-shaped hole and the other with a star-shaped hole, and then matched them with two nonsense words. The curvature profile of the hole boundary was manipulated so that the shape of the interior region (i.e., a hole) and that of the exterior region (i.e., material edges) give rise to opposite shape impressions (i.e., one rounded and the other spiky). The results of Experiment 1 revealed that shape-name matching for holed objects is based on the interior shapes of holes, but not those of materially defined inner edges. The following three experiments replicated the same pattern of results even when holes appeared like oral apertures in animal character faces (Experiments 2-3) and when they were irregular, non-symmetric, and low in semantic association with familiar real-world objects (Experiment 4). Lastly, Experiment 5 showed that shape-name matching for "C"-shaped, negative-part stimuli is also interior-shape-based if the opening of the interior region is relatively small. These findings suggest that the interior shapes of holes are automatically accessible. I conclude with a discussion of my hypothesis that the only global-level, unitary shape representation of a bounded region of a single connected surface is that of the interior region for both objects and holes, imposing an important constraint in visual shape processing.

Cross-modal correspondences in sine wave: Speech versus non-speech modes

The present study aimed to investigate whether or not the so-called "bouba-kiki" effect is mediated by speech-specific representations. Sine-wave versions of naturally produced pseudowords were used as auditory stimuli in an implicit association task (IAT) and an explicit cross-modal matching (CMM) task to examine cross-modal shape-sound correspondences. A group of participants trained to hear the sine-wave stimuli as speech was compared to a group that heard them as non-speech sounds. Sound-shape correspondence effects were observed in both groups and tasks, indicating that speech-specific processing is not fundamental to the "bouba-kiki" phenomenon. Effects were similar across groups in the IAT, while in the CMM task the speech-mode group showed a stronger effect compared with the non-speech group. This indicates that, while both tasks reflect auditory-visual associations, only the CMM task is additionally sensitive to associations involving speech-specific representations.