Shape-color associations in an unrestricted color choice paradigm

Since Kandinsky's claim for fundamental shape-color associations, several studies have revealed that those tendencies were not generalizable to the entire population and that different associations were more prevalent. Past studies, however, lacked a methodology that allowed participants to freely report their shape-color preferences. Here, we report data from 7,517 Danish individuals, using a free choice full color wheel for five different geometrical shapes. We find significant shape-hue associations for circle-red/yellow, triangle-green/yellow, square-blue, and pentagon/hexagon-magenta. The significant shape-hue associations are also more saturated than non-significant ones for the circle, triangle, and square. At the conceptual level, basic shapes, which show stronger associations, are linked to primary colors, and non-basic shapes to secondary colors. Shape-color associations seem indeed to follow the Berlin-Kay stages of entry into languages. This pattern had previously been described for graphemes and weekday-color associations. The methodology employed in our study can be repeated in different cultural contexts in the future. We also provide another instance of color associations for ordinal concepts that follow the stages of entry into languages.

Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution

Introduction

Visual-to-auditory sensory substitution devices are assistive devices for the blind that convert visual images into auditory images (or soundscapes) by mapping visual features with acoustic cues. To convey spatial information with sounds, several sensory substitution devices use a Virtual Acoustic Space (VAS) using Head Related Transfer Functions (HRTFs) to synthesize natural acoustic cues used for sound localization. However, the perception of the elevation is known to be inaccurate with generic spatialization since it is based on notches in the audio spectrum that are specific to each individual. Another method used to convey elevation information is based on the audiovisual cross-modal correspondence between pitch and visual elevation. The main drawback of this second method is caused by the limitation of the ability to perceive elevation through HRTFs due to the spectral narrowband of the sounds.

Method

In this study we compared the early ability to localize objects with a visual-to-auditory sensory substitution device where elevation is either conveyed using a spatialization-based only method (Noise encoding) or using pitch-based methods with different spectral complexities (Monotonic and Harmonic encodings). Thirty eight blindfolded participants had to localize a virtual target using soundscapes before and after having been familiarized with the visual-to-auditory encodings.

Results

Participants were more accurate to localize elevation with pitch-based encodings than with the spatialization-based only method. Only slight differences in azimuth localization performance were found between the encodings.

Discussion

This study suggests the intuitiveness of a pitch-based encoding with a facilitation effect of the cross-modal correspondence when a non-individualized sound spatialization is used.

Complex Shapes Are Bluish, Darker, and More Saturated; Shape-Color Correspondence in 3D Object Perception

It has been shown that there is a non-random association between shape and color. However, the results of previous studies on the shape-color correspondence did not converge. To address the issue, we focused on shape complexity among a number of shape properties, particularly in terms of 3D shape, and parametrically manipulated the shape complexity and all three components of color. With two experiments, the current study aimed to closely examine the correspondence between shape complexity of 3D shape and color in terms of hue (Experiment 1), luminance, and saturation (Experiment 2). Participants were presented with the 3D shapes in either visual or visuo-haptic modes of exploration. Subsequently, they had to pick from a color palette the color best matching each shape of the object. In Experiment 1, we found that as shapes became more complex, the best associated hue changed from those with long wavelengths to ones with short wavelengths. Results of Experiment 2 demonstrated that as the shapes grew more complex, the associated luminance decreased, and saturation increased. Additionally, adding haptic exploration to visual exploration strengthened the association – for saturation in particular – with the pattern of shape-color correspondence maintained. Taken together, we demonstrated that complex shapes are associated with bluish, darker and more saturated colors, suggesting that shape complexity has a systematic relationship with color including hue, luminance, and saturation.

The Communication of Timbral Intentions Between Pianists and Listeners and Its Dependence on Auditory-Visual Conditions

The perceptual experiment reported in this article explored whether the communication of five pairs of timbral intentions (bright/dark, heavy/light, round/sharp, tense/relaxed, and dry/velvety) between pianists and listeners is reliable and the extent to which performers' gestures provide visual cues that influence the perceived timbre. Three pianists played three musical excerpts with 10 different timbral intentions (3 × 10 = 30 music stimuli) and 21 piano students were asked to rate perceived timbral qualities on both unipolar Likert scales and non-verbal sensory scales (shape, size, and brightness) under three modes (vision-alone, audio-alone, and audio-visual). The results revealed that nine of the timbral intentions were reliably communicated between the pianists and the listeners, except for the dark timbre. The communication of tense and relaxed timbres was improved by the visual conditions regardless of who is performing; for the rest, we found the individuality in each pianist's preference for using visual cues. The results also revealed a strong cross-modal association between timbre and shape. This study implies that the communication of piano timbre is not based on acoustic cues alone but relates to a shared understanding of sensorimotor experiences between the performers and the listeners.

Sooo Sweeet! Presence of Long Vowels in Brand Names Lead to Expectations of Sweetness

Throughout the history of languages, poets and writers have used linguistic tools to enhance euphony in their creations. One of the widely used tools to convey melody in any written (or spoken) creative art form is the use of long vowels. This paper examines the linkages between long (vs. short) vowel sounds and taste expectations of sweetness. Across four studies, we demonstrate that people expect products with brand names containing long vowels to taste sweeter than those including short vowel sounds. In studies 1 and 2, we demonstrate this association with the use of self-reported measures, and in studies 3 and 4, we employ indirect measures (implicit taste-shape correspondence and Single Category Implicit Association Test (SC-IAT) paradigm) to show the effect holds at a subconscious level of processing. Previous research in this field has typically linked vowel position (high vs. low or front vs. back) with product or brand attribute expectations. This paper contributes to the growing body of literature in this field by demonstrating the importance of vowel length in sound symbolism, and more precisely, how it pertains to the taste continuum.

Visual and Proprioceptive Perceptions Evoke Motion-Sound Symbolism: Different Acceleration Profiles Are Associated With Different Types of Consonants

A growing body of literature has shown that one perceptual modality can be systematically associated with sensation in another. However, the cross-modal relationship between linguistic sounds and motions (i.e., motion-sound symbolism) is an extremely understudied area of research. Against this background, this paper examines the cross-modal correspondences between categories of consonants on one hand and different acceleration profiles of motion stimuli on the other. In the two experiments that we conducted, we mechanically manipulated the acceleration profiles of the stimuli while holding the trajectory paths constant, thus distinguishing the effect of acceleration profiles from that of motion path shapes. The results show that different acceleration profiles can be associated with different types of consonants; in particular, movements with acceleration and deceleration tend to be associated with a class of sounds called obstruents, whereas movements without much acceleration tend to be associated with a class of sounds called sonorants. Moreover, the current experiments show that this sort of cross-modal correspondence arises even when the stimuli are not presented visually, namely, when the participants' hands were moved passively by a manipulandum. In conclusion, the present study adds an additional piece of evidence demonstrating that bodily action-based information, i.e., proprioception as a very feasible candidate, could lead to sound symbolic patterns.

Cross-Modal Correspondences Between Temperature and Taste Attributes

Temperature is an important characteristic of food and drink. In addition to food-intrinsic temperature (i.e., serving temperature), consumers often experience food-extrinsic temperature (e.g., physical warmth). Emerging research on cross-modal correspondence has revealed that people reliably associate temperature with other sensory features. Building on the literature on cross-modal correspondence and sensation transference theory, the present study aimed to reveal mental representations of temperature–taste correspondence and cross-modal mental representations influencing corresponding sensory/hedonic perceptions of beverages, with a focus on manipulating food-extrinsic warmth. To reveal mental representations of temperature–taste correspondence, Experiment 1 investigated whether temperature words (warm, cool) are associated with sensory/hedonic attributes (e.g., sweet, sour, salty, bitter). The results of Experiment 1 demonstrated that warm (vs. cool) was matched more with saltiness, tastiness, healthfulness, and preference (intention to buy), whereas cool (vs. warm) was matched more with sourness and freshness. Experiment 2 assessed whether cross-modal mental representations influence corresponding sensory/hedonic perceptions of beverages. The participants wore hot and cold pads and rated sensory/hedonic attributes of Japanese tea (Experiment 2a) or black coffee (Experiment 2b) before and after tasting it. The results of Experiment 2a demonstrated that physical warmth (vs. coldness) increased healthfulness and the intention to buy Japanese tea. The results of Experiment 2b did not reveal any effects of physical warmth on sensory/hedonic ratings. These findings provide evidence of taste–temperature correspondence and provide preliminary support for the influence of food-extrinsic warmth on taste attributes related to positivity.

A Protracted Sensitive Period Regulates the Development of Cross-Modal Sound-Shape Associations in Humans

Humans preferentially match arbitrary words containing higher- and lower-frequency phonemes to angular and smooth shapes, respectively. Here, we investigated the role of visual experience in the development of audiovisual and audiohaptic sound-shape associations (SSAs) using a unique set of five groups: individuals who had suffered a transient period of congenital blindness through congenital bilateral dense cataracts before undergoing cataract-reversal surgeries (CC group), individuals with a history of developmental cataracts (DC group), individuals with congenital permanent blindness (CB group), individuals with late permanent blindness (LB group), and controls with typical sight (TS group). Whereas the TS and LB groups showed highly robust SSAs, the CB, CC, and DC groups did not-in any of the modality combinations tested. These results provide evidence for a protracted sensitive period during which aberrant vision prevents SSA acquisition. Moreover, the finding of a systematic SSA in the LB group demonstrates that representations acquired during the sensitive period are resilient to loss despite dramatically changed experience.

Hand-Feel Touch Cues and Their Influences on Consumer Perception and Behavior with Respect to Food Products: A Review

There has been a great deal of research investigating intrinsic/extrinsic cues and their influences on consumer perception and purchasing decisions at points of sale, product usage, and consumption. Consumers create expectations toward a food product through sensory information extracted from its surface (intrinsic cues) or packaging (extrinsic cues) at retail stores. Packaging is one of the important extrinsic cues that can modulate consumer perception, liking, and decision making of a product. For example, handling a product packaging during consumption, even just touching the packaging while opening or holding it during consumption, may result in a consumer expectation of the package content. Although hand-feel touch cues are an integral part of the food consumption experience, as can be observed in such an instance, little has been known about their influences on consumer perception, acceptability, and purchase behavior of food products. This review therefore provided a better understanding about hand-feel touch cues and their influences in the context of food and beverage experience with a focus on (1) an overview of touch as a sensory modality, (2) factors influencing hand-feel perception, (3) influences of hand-feel touch cues on the perception of other sensory modalities, and (4) the effects of hand-feel touch cues on emotional responses and purchase behavior.

Introduction to the Special Issue on Multisensory Research Forum (IMRF 2016, Suzhou)

This editorial introduction briefly summarized the general information and keynote topics of 17th International Multisensory Research Forum (IMRF 2016), which was held in Suzhou, China on June 15-18, 2016. In this IMRF2016 Multisensory Research special issue, seven papers have been collected. Some topics represent mainstream, traditional studies, while others show a range of multidisciplinary approaches. We categorized the main topics in this issue to four aspects: expertise in multisensory timing, neuronal signatures underlying cross-modal correspondence, coupling between perception and action, and lastly, a revisit of the principles for multisensory segregation and integration.

[i] is Lighter and More Greenish Than [o]: Intrinsic Association Between Vowel Sounds and Colors

It has recently been reported in the synesthesia literature that graphemes sharing the same phonetic feature tend to induce similar synesthetic colors. In the present study, we investigated whether phonetic properties are associated with colors in a specific manner among the general population, even when other visual and linguistic features of graphemes are removed. To test this hypothesis, we presented vowel sounds synthesized by systematically manipulating the position of the tongue body's center. Participants were asked to choose a color after hearing each sound. Results from the main experiment showed that lightness and chromaticity of matched colors exhibited systematic variations along the two axes of the position of the tongue body's center. Some non-random associations between vowel sounds and colors remained effective with pitch and intensity of the sounds equalized in the control experiment, which suggests that other acoustic factors such as inherent pitch of vowels cannot solely account for the current results. Taken together, these results imply that the association between phonetic features and colors is not random, and this synesthesia-like association is shared by people in the general population.

Iconicity in the lab: a review of behavioral, developmental, and neuroimaging research into sound-symbolism

This review covers experimental approaches to sound-symbolism—from infants to adults, and from Sapir’s foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides us with opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language.

Ideophones in Japanese modulate the P2 and late positive complex responses

Sound-symbolism, or the direct link between sound and meaning, is typologically and behaviorally attested across languages. However, neuroimaging research has mostly focused on artificial non-words or individual segments, which do not represent sound-symbolism in natural language. We used EEG to compare Japanese ideophones, which are phonologically distinctive sound-symbolic lexical words, and arbitrary adverbs during a sentence reading task. Ideophones elicit a larger visual P2 response than arbitrary adverbs, as well as a sustained late positive complex. Our results and previous literature suggest that the larger P2 may indicate the integration of sound and sensory information by association in response to the distinctive phonology of ideophones. The late positive complex may reflect the facilitated lexical retrieval of arbitrary words in comparison to ideophones. This account provides new evidence that ideophones exhibit similar cross-modal correspondences to those which have been proposed for non-words and individual sounds.

It does belong together: cross-modal correspondences influence cross-modal integration during perceptual learning

Experiencing a stimulus in one sensory modality is often associated with an experience in another sensory modality. For instance, seeing a lemon might produce a sensation of sourness. This might indicate some kind of cross-modal correspondence between vision and gustation. The aim of the current study was to explore whether such cross-modal correspondences influence cross-modal integration during perceptual learning. To that end, we conducted two experiments. Using a speeded classification task, Experiment 1 established a cross-modal correspondence between visual lightness and the frequency of an auditory tone. Using a short-term priming procedure, Experiment 2 showed that manipulation of such cross-modal correspondences led to the creation of a crossmodal unit regardless of the nature of the correspondence (i.e., congruent, Experiment 2a or incongruent, Experiment 2b). However, a comparison of priming effects sizes suggested that cross-modal correspondences modulate cross-modal integration during learning, leading to new learned units that have different stability over time. We discuss the implications of our results for the relation between cross-modal correspondence and perceptual learning in the context of a Bayesian explanation of cross-modal correspondences.

Lower pitch is larger, yet falling pitches shrink

Experiments using diverse paradigms, including speeded discrimination, indicate that pitch and visually-perceived size interact perceptually, and that higher pitch is congruent with smaller size. While nearly all of these studies used static stimuli, here we examine the interaction of dynamic pitch and dynamic size, using Garner's speeded discrimination paradigm. Experiment 1 examined the interaction of continuous rise/fall in pitch and increase/decrease in object size. Experiment 2 examined the interaction of static pitch and size (steady high/low pitches and large/small visual objects), using an identical procedure. Results indicate that static and dynamic auditory and visual stimuli interact in opposite ways. While for static stimuli (Experiment 2), higher pitch is congruent with smaller size (as suggested by earlier work), for dynamic stimuli (Experiment 1), ascending pitch is congruent with growing size, and descending pitch with shrinking size. In addition, while static stimuli (Experiment 2) exhibit both congruence and Garner effects, dynamic stimuli (Experiment 1) present congruence effects without Garner interference, a pattern that is not consistent with prevalent interpretations of Garner's paradigm. Our interpretation of these results focuses on effects of within-trial changes on processing in dynamic tasks and on the association of changes in apparent size with implied changes in distance. Results suggest that static and dynamic stimuli can differ substantially in their cross-modal mappings, and may rely on different processing mechanisms.

Natural auditory scene statistics shapes human spatial hearing

Human perception, cognition, and action are laced with seemingly arbitrary mappings. In particular, sound has a strong spatial connotation: Sounds are high and low, melodies rise and fall, and pitch systematically biases perceived sound elevation. The origins of such mappings are unknown. Are they the result of physiological constraints, do they reflect natural environmental statistics, or are they truly arbitrary? We recorded natural sounds from the environment, analyzed the elevation-dependent filtering of the outer ear, and measured frequency-dependent biases in human sound localization. We find that auditory scene statistics reveals a clear mapping between frequency and elevation. Perhaps more interestingly, this natural statistical mapping is tightly mirrored in both ear-filtering properties and in perceived sound location. This suggests that both sound localization behavior and ear anatomy are fine-tuned to the statistics of natural auditory scenes, likely providing the basis for the spatial connotation of human hearing.

Color and texture associations in voice-induced synesthesia

Voice-induced synesthesia, a form of synesthesia in which synesthetic perceptions are induced by the sounds of people's voices, appears to be relatively rare and has not been systematically studied. In this study we investigated the synesthetic color and visual texture perceptions experienced in response to different types of “voice quality” (e.g., nasal, whisper, falsetto). Experiences of three different groups—self-reported voice synesthetes, phoneticians, and controls—were compared using both qualitative and quantitative analysis in a study conducted online. Whilst, in the qualitative analysis, synesthetes used more color and texture terms to describe voices than either phoneticians or controls, only weak differences, and many similarities, between groups were found in the quantitative analysis. Notable consistent results between groups were the matching of higher speech fundamental frequencies with lighter and redder colors, the matching of “whispery” voices with smoke-like textures, and the matching of “harsh” and “creaky” voices with textures resembling dry cracked soil. These data are discussed in the light of current thinking about definitions and categorizations of synesthesia, especially in cases where individuals apparently have a range of different synesthetic inducers.