Crossmodal Correspondences in the Sounds of Chinese Instruments

Color and tone color: audiovisual crossmodal correspondences with musical instrument timbre

Crossmodal correspondences, or widely shared tendencies for mapping experiences across sensory domains, are revealed in common descriptors of musical timbre such as bright, dark, and warm. Two experiments are reported in which participants listened to recordings of musical instruments playing major scales, selected colors to match the timbres, and rated the timbres on crossmodal semantic scales. Experiment A used three different keyboard instruments, each played in three pitch registers. Stimuli in Experiment B, representing six different orchestral instruments, were similar to those in Experiment A but were controlled for pitch register. Overall, results were consistent with hypothesized concordances between ratings on crossmodal timbre descriptors and participants' color associations. Semantic ratings predicted the lightness and saturation of colors matched to instrument timbres; effects were larger when both pitch register and instrument type varied (Experiment A) but were still evident when pitch register was held constant (Experiment B). We also observed a weak relationship between participant ratings of musical stimuli on the terms warm and cool and the warmth-coolness of selected colors in Experiment B only. Results were generally consistent with the hypothesis that instrument type and pitch register are related to color choice, though we speculate that these associations may only be relevant for certain instruments. Overall, the results have implications for our understanding the relationship between music and color, suggesting that while timbre/color matching behavior is in many ways diverse, observable trends in strategy can in part be linked to crossmodal timbre semantics.

Inducing Novel Sound-Taste Correspondences via an Associative Learning Task

The interest in crossmodal correspondences, including those involving sounds and involving tastes, has experienced rapid growth in recent years. However, the mechanisms underlying these correspondences are not well understood. In the present study (N = 302), we used an associative learning paradigm, based on previous literature using simple sounds with no consensual taste associations (i.e., square and triangle wave sounds at 200 Hz) and taste words (i.e., sweet and bitter), to test the influence of two potential mechanisms in establishing sound-taste correspondences and investigate whether either learning mechanism could give rise to new and long-lasting associations. Specifically, we examined an emotional mediation account (i.e., using sad and happy emoji facial expressions) and a transitive path (i.e., sound-taste correspondence being mediated by color, using red and black colored squares). The results revealed that the associative learning paradigm mapping the triangle wave tone with a happy emoji facial expression induced a novel crossmodal correspondence between this sound and the word sweet. Importantly, we found that this novel association was still present two months after the experimental learning paradigm. None of the other mappings, emotional or transitive, gave rise to any significant associations between sound and taste. These findings provide evidence that new crossmodal correspondences between sounds and tastes can be created by leveraging the affective connection between both dimensions, helping elucidate the mechanisms underlying these associations. Moreover, these findings reveal that these associations can last for several weeks after the experimental session through which they were induced.

The taste & affect music database: Subjective rating norms for a new set of musical stimuli

Music is a ubiquitous stimulus known to influence human affect, cognition, and behavior. In the context of eating behavior, music has been associated with food choice, intake and, more recently, taste perception. In the latter case, the literature has reported consistent patterns of association between auditory and gustatory attributes, suggesting that individuals reliably recognize taste attributes in musical stimuli. This study presents subjective norms for a new set of 100 instrumental music stimuli, including basic taste correspondences (sweetness, bitterness, saltiness, sourness), emotions (joy, anger, sadness, fear, surprise), familiarity, valence, and arousal. This stimulus set was evaluated by 329 individuals (83.3% women; Mage = 28.12, SD = 12.14), online (n = 246) and in the lab (n = 83). Each participant evaluated a random subsample of 25 soundtracks and responded to self-report measures of mood and taste preferences, as well as the Goldsmiths Musical Sophistication Index (Gold-MSI). Each soundtrack was evaluated by 68 to 97 participants (Mdn = 83), and descriptive results (means, standard deviations, and confidence intervals) are available as supplemental material at osf.io/2cqa5 . Significant correlations between taste correspondences and emotional/affective dimensions were observed (e.g., between sweetness ratings and pleasant emotions). Sex, age, musical sophistication, and basic taste preferences presented few, small to medium associations with the evaluations of the stimuli. Overall, these results suggest that the new Taste & Affect Music Database is a relevant resource for research and intervention with musical stimuli in the context of crossmodal taste perception and other affective, cognitive, and behavioral domains.