Crossmodal correspondences between odors and contingent features: odors, musical notes, and geometrical shapes

Crossmodal associations between naturally occurring tactile and sound textures

This study investigates the crossmodal associations between naturally occurring sound textures and tactile textures. Previous research has demonstrated the association between low-level sensory features of sound and touch, as well as higher-level, cognitively mediated associations involving language, emotions, and metaphors. However, stimuli like textures, which are found in both modalities have received less attention. In this study, we conducted two experiments: a free association task and a two alternate forced choice task using everyday tactile textures and sound textures selected from natural sound categories. The results revealed consistent crossmodal associations reported by participants between the textures of the two modalities. They tended to associate more sound textures (e.g., wood shavings and sandpaper) with tactile surfaces that were rated as harder, rougher, and intermediate on the sticky-slippery scale. While some participants based the auditory-tactile association on sensory features, others made the associations based on semantic relationships, co-occurrence in nature, and emotional mediation. Interestingly, the statistical features of the sound textures (mean, variance, kurtosis, power, autocorrelation, and correlation) did not show significant correlations with the crossmodal associations, indicating a higher-level association. This study provides insights into auditory-tactile associations by highlighting the role of sensory and emotional (or cognitive) factors in prompting these associations.

Enhancing the design of wine labels

Introduction

The knowledge accrued through research in the domain of crossmodal correspondences has had a significant influence on a diverse array of disciplines, including behavioral studies, neuroscience, computational modeling, and notably, marketing, with the objective of aligning sensory experiences to help shape patterns of consumer behavior. A study is reported that explores the extension of these principles to the communication of products having a notably complex sensory profile, specifically within the context of wine. The central aim of the project is to explore the feasibility of using crossmodal communication as a strategic tool to augment the congruence between the consumers' multisensory expectations and their sensory experiences. For consumers venturing into the realm of wine selection without the advantage of prior tasting experience, it is of paramount importance to possess a robust understanding of the mandated information. This encompasses critical elements such as the wine's origin, grape varietal(s) used, geographical indications, producer qualifications, and the potential implications of these factors on the final wine product. This level of comprehension stands as a necessary prerequisite, enabling these consumers to make informed choices that align with their preferences, even in the absence of previous sensory encounters. Nonetheless, semiotic investigations underscore the significance attributed to symbolic components such as signs, logos, colors, gestures, and linguistic cues. Research from the field performing multisensory studies, presents a counterpoint to prevailing communication paradigms, advocating for a heightened incorporation of metaphors, analogies, symbols, metonymies, and allegories. This alternative approach aims to enhance the efficacy of communication strategies, offering a more profound and evocative means of conveying intricate messages on a more holistic level.

Methods

A questionnaire was sent to a specific group of engaged wine consumers (n = 329). Besides questions regarding demographics, purchase behavior, and consumption behavior, the questionnaire included examples of multisensory communication through a selection of symbols, as well as alternative wine information.

Results

The results showed significant correlations between demographics, consumption behavior, and attitudes toward the tested multisensory symbols and alternative information, thus helping to gain a better understanding of the sensory properties that should be communicated on wine labels.

Discussion

The findings reported here highlight the effectiveness of visual crossmodal communication as a promising pathway capable of skillfully capturing consumer attributes, conveying multisensory experiences, and portraying the comprehensive timeline of taste evolution. As a result, it assumes a pivotal role as a communicative tool for intricate consumables, like wine, functioning at the crossroads of visual and sensory dimensions.

Crossmodal associations between visual textures and temperature concepts

Visual textures are critical in how individuals form sensory expectations about objects, which include somatosensory properties such as temperature. This study aimed to uncover crossmodal associations between visual textures and temperature concepts. In Experiment 1 (N = 193), we evaluated crossmodal associations between 43 visual texture categories and different temperature concepts (via temperature words such as cold and hot) using an explicit forced-choice test. The results revealed associations between striped, cracked, matted, and waffled visual textures and high temperatures and between crystalline and flecked visual textures and low temperatures. In Experiment 2 (N = 247), we conducted six implicit association tests (IATs) pairing the two visual textures most strongly associated with low (crystalline and flecked) and high (striped and cracked) temperatures with the words cold and hot as per the results of Experiment 1. When pairing the crystalline and striped visual textures, the results revealed that crystalline was matched to the word cold, and striped was matched to the word hot. However, some associations found in the explicit test were not found in the IATs. In Experiment 3 (N = 124), we investigated how mappings between visual textures and concrete entities may influence crossmodal associations with temperature and these visual textures. Altogether, we found a range of association strengths and automaticity levels. Importantly, we found evidence of relative effects. Furthermore, some of these crossmodal associations are partly influenced by indirect mappings to concrete entities.

Shared mental representations underlie metaphorical sound concepts

Communication between sound and music experts is based on the shared understanding of a metaphorical vocabulary derived from other sensory modalities. Yet, the impact of sound expertise on the mental representation of these sound concepts remains blurry. To address this issue, we investigated the acoustic portraits of four metaphorical sound concepts (brightness, warmth, roundness, and roughness) in three groups of participants (sound engineers, conductors, and non-experts). Participants (N = 24) rated a corpus of orchestral instrument sounds (N = 520) using Best-Worst Scaling. With this data-driven method, we sorted the sound corpus for each concept and population. We compared the population ratings and ran machine learning algorithms to unveil the acoustic portraits of each concept. Overall, the results revealed that sound engineers were the most consistent. We found that roughness is widely shared while brightness is expertise dependent. The frequent use of brightness by expert populations suggests that its meaning got specified through sound expertise. As for roundness and warmth, it seems that the importance of pitch and noise in their acoustic definition is the key to distinguishing them. These results provide crucial information on the mental representations of a metaphorical vocabulary of sound and whether it is shared or refined by sound expertise.

Learning-Induced Odor Modulation of Neuronal Activity in Auditory Cortex

Sensory cortices, even of primary regions, are not purely unisensory. Rather, cortical neurons in sensory cortex show various forms of multisensory interactions. While some multisensory interactions naturally co-occur, the combination of others will co-occur through experience. In real life, learning and experience will result in conjunction with seemingly disparate sensory information that ultimately becomes behaviorally relevant, impacting perception, cognition, and action. Here we describe a novel auditory discrimination task in mice, designed to manipulate the expectation of upcoming trials using olfactory cues. We show that, after learning, female mice display a transient period of several days during which they exploit odor-mediated expectations for making correct decisions. Using two-photon calcium imaging of single neurons in auditory cortex (ACx) during behavior, we found that the behavioral effects of odor-mediated expectations are accompanied by an odor-induced modulation of neuronal activity. Further, we find that these effects are manifested differentially, based on the response preference of individual cells. A significant portion of effects, but not all, are consistent with a predictive coding framework. Our data show that learning novel odor-sound associations evoke changes in ACx. We suggest that behaviorally relevant multisensory environments mediate contextual effects as early as ACx.SIGNIFICANCE STATEMENT Natural environments are composed of multisensory objects. It remains unclear whether and how animals learn the regularities of congruent multisensory associations and how these may impact behavior and neural activity. We tested how learned odor-sound associations affected single-neuron responses in auditory cortex. We introduce a novel auditory discrimination task for mice in which odors set different contexts of expectation to upcoming trials. We show that, although the task can be solved purely by sounds, odor-mediated expectation impacts performance. We further show that odors cause a modulation of neuronal activity in auditory cortex, which is correlated with behavior. These results suggest that learning prompts an interaction of odor and sound information as early as sensory cortex.

Olfactory abstraction: a communicative and metacognitive account

The usual puzzle raised about olfaction is that of a deficit of abstraction: smells, by contrast notably with colours, do not easily lend themselves to abstract categories and labels. Some studies have argued that the puzzle is culturally restricted and that abstraction is more common outside urban Western societies. Here, I argue that the puzzle is misconstrued and should be reversed: given that odours are constantly changing and that their commonalities are difficult for humans to identify, what is surprising is not that abstract terms are rare, but that they should be used at all for olfaction. Given the nature of the olfactory environment and our cognitive equipment, concrete labels referring to sources seem most adaptive. To explain the use and presence of abstract terms, we need to examine their social and communicative benefits. Here these benefits are spelt out as securing a higher agreement among individuals varying in their olfactory experiences as well as the labels they use, as well as feeling a heightened sense of confidence in one's naming capacities. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Searching for perceptual similarity within, and between, the (chemical) senses

In this narrative historical review, I want to take a closer look at the concept of perceptual similarity both as it applies within, and between, the chemical senses (specifically taste and smell). The discussion is linked to issues of affective similarity and connotative meaning. The relation between intramodal and crossmodal judgments of perceptual similarity, and the putatively special status of those odorants that happen to take on taste qualities will also be discussed. An important distinction is drawn between the interrelated, though sometimes distinct, notions of perceptual similarity and crossmodal congruency, specifically as they relate to the comparison of chemosensory stimuli. Such phenomena are often referred to as crossmodal correspondences, or by others (incorrectly in my view), as a kind of ubiquitous synesthesia.

Roughness perception: A multisensory/crossmodal perspective

Roughness is a perceptual attribute typically associated with certain stimuli that are presented in one of the spatial senses. In auditory research, the term is typically used to describe the harsh effects that are induced by particular sound qualities (i.e., dissonance) and human/animal vocalizations (e.g., screams, distress cries). In the tactile domain, roughness is a crucial factor determining the perceptual features of a surface. The same feature can also be ascertained visually, by means of the extraction of pattern features that determine the haptic quality of surfaces, such as grain size and density. By contrast, the term roughness has rarely been applied to the description of those stimuli perceived via the chemical senses. In this review, we take a critical look at the putative meaning(s) of the term roughness, when used in both unisensory and multisensory contexts, in an attempt to answer two key questions: (1) Is the use of the term 'roughness' the same in each modality when considered individually? and (2) Do crossmodal correspondences involving roughness match distinct perceptual features or (at least on certain occasions) do they merely pick-up on an amodal property? We start by examining the use of the term in the auditory domain. Next, we summarize the ways in which the term roughness has been used in the literature on tactile and visual perception, and in the domain of olfaction and gustation. Then, we move on to the crossmodal context, reviewing the literature on the perception of roughness in the audiovisual, audiotactile, and auditory-gustatory/olfactory domains. Finally, we highlight some limitations of the reviewed literature and we outline a number of key directions for future empirical research in roughness perception.

Unpleasant Odors Affect Alerting Attention in Young Men: An Event-Related Potential Study Using the Attention Network Test

Attention to unpleasant odors is crucial for human safety because they may signal danger; however, whether odor concentration also plays a role remains debated. Here, we explored the effects of two concentrations of pleasant and unpleasant odors on the attention network, comprising the alerting, orienting, and executive control networks. Behavioral responses were examined using the Attention Network Test, while electrophysiological responses were examined by assessing N1 and N2 amplitudes in 30 young men. We found that irrespective of odor concentration, an unpleasant odor induced larger cue-related N1 and N2 amplitudes in the alerting and executive control networks at occipital and frontal electrode sites and that was only paralleled by a reduced behavioral response time of cue-related trails in the alerting network. Thus, our results do not provide supporting evidence for a concentration-dependent effect, but they do suggest that more attentional resources are allocated to alerting-relevant stimuli to improve behavioral response times to a potential threat in young men.

Crossmodal Associations with Olfactory, Auditory, and Tactile Stimuli in Children and Adults

People associate information with different senses but the mechanism by which this happens is unclear. Such associations are thought to arise from innate structural associations in the brain, statistical associations in the environment, via shared affective content, or through language. A developmental perspective on crossmodal associations can help determine which explanations are more likely for specific associations. Certain associations with pitch (e.g., pitch-height) have been observed early in infancy, but others may only occur late into childhood (e.g., pitch-size). In contrast, tactile-chroma associations have been observed in children, but not adults. One modality that has received little attention developmentally is olfaction. In the present investigation, we explored crossmodal associations from sound, tactile stimuli, and odor to a range of stimuli by testing a broad range of participants. Across the three modalities, we found little evidence for crossmodal associations in young children. This suggests an account based on innate structures is unlikely. Instead, the number and strength of associations increased over the lifespan. This suggests that experience plays a crucial role in crossmodal associations from sound, touch, and smell to other senses.

Crossmodal Interactions Between Olfaction and Touch Affecting Well-Being and Perception of Cosmetic Creams

In the present series of studies, we investigated crossmodal perception of odor and texture. In four studies, participants tried two textures of face creams, one high viscosity (HV) and one low viscosity (LV), each with one of three levels of added odor (standard level, half of standard, or base [no added odor]), and then reported their levels of well-being. They also reported their perceptions of the face creams, including liking (global liking of the product, liking of its texture) and “objective” evaluations on just about right (JAR) scales (texture and visual appearance evaluations). In Study 1, women in France tried the creams on their hands, as they would when testing them in a store, and in Study 2, a second group of French women tried the creams on their faces, as they would at home. In Studies 3 and 4, these same two procedures were repeated in China. Results showed that both odor and texture had effects on well-being, liking, and JAR ratings, including interaction effects. Though effects varied by country and context (hand or face), the addition of odor to the creams generally increased reports of well-being, global liking and texture liking, in some cases affecting the “objective” evaluations of texture. This is one of the first investigations of crossmodal olfactory and tactile perception's impacts on well-being, and it reinforces previous literature showing the importance of olfaction on well-being.

Musical Scents: On the Surprising Absence of Scented Musical/Auditory Events, Entertainments, and Experiences

The matching of scents with music is both one of the most natural (or intuitive) of crossmodal correspondences and, at the same time, one of the least frequently explored combinations of senses in an entertainment and multisensory experiential design context. This narrative review highlights the various occasions over the last century or two when scents and sounds have coincided, and the various motivations behind those who have chosen to bring these senses together: This has included everything from the masking of malodour to the matching of the semantic meaning or arousal potential of the two senses, through to the longstanding and recently-reemerging interest in the crossmodal correspondences (now that they have been distinguished from the superficially similar phenomenon of synaesthesia, with which they were previously often confused). As such, there exist a number of ways in which these two senses can be incorporated into meaningful multisensory experiences that can potentially resonate with the public. Having explored the deliberate combination of scent and music (or sound) in everything from "scent-sory" marketing through to fragrant discos and olfactory storytelling, I end by summarizing some of the opportunities around translating such unusual multisensory experiences from the public to the private sphere. This will likely be via the widespread dissemination of sensory apps that promise to convert (or translate) from one sense (likely scent) to another (e.g., music), as has, for example already started to occur in the world of music selections to match the flavour of specific wines.

May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality

When interacting with technology, attention is mainly driven by audiovisual and increasingly haptic stimulation. Olfactory stimuli are widely neglected, although the sense of smell influences many of our daily life choices, affects our behavior, and can catch and direct our attention. In this study, we investigated the effect of smell and sound on visuospatial attention in a virtual environment. We implemented the Bells Test, an established neuropsychological test to assess attentional and visuospatial disorders, in virtual reality (VR). We conducted an experiment with 24 participants comparing the performance of users under three experimental conditions (smell, sound, and smell and sound). The results show that multisensory stimuli play a key role in driving the attention of the participants and highlight asymmetries in directing spatial attention. We discuss the relevance of the results within and beyond human-computer interaction (HCI), particularly with regard to the opportunity of using VR for rehabilitation and assessment procedures for patients with spatial attention deficits.

The scent of attraction and the smell of success: crossmodal influences on person perception

In recent decades, there has been an explosion of research into the crossmodal influence of olfactory cues on multisensory person perception. Numerous peer-reviewed studies have documented that a variety of olfactory stimuli, from ambient malodours through to fine fragrances, and even a range of chemosensory body odours can influence everything from a perceiver's judgments of another person's attractiveness, age, affect, health/disease status, and even elements of their personality. The crossmodal and multisensory contributions to such effects are reviewed and the limitations/peculiarities of the research that have been published to date are highlighted. At the same time, however, it is important to note that the presence of scent (and/or the absence of malodour) can also influence people's (i.e., a perceiver's) self-confidence which may, in turn, affect how attractive they appear to others. Several potential cognitive mechanisms have been put forward to try and explain such crossmodal/multisensory influences, and some of the neural substrates underpinning these effects have now been characterized. At the end of this narrative review, a number of the potential (and actual) applications for, and implications of, such crossmodal/multisensory phenomena involving olfaction are outlined briefly.

Explaining Crossmodal Correspondences Between Colours and Tastes

For centuries, if not millennia, people have associated the basic tastes (e.g., sweet, bitter, salty, and sour) with specific colours. While the range of tastes may have changed, and the reasons for wanting to connect the senses in this rather surprising way have undoubtedly differed, there would nevertheless appear to be a surprisingly high degree of consistency regarding this crossmodal mapping among non-synaesthetes that merits further consideration. Traditionally, colour-taste correspondences have often been considered together with odour-colour and flavour-colour correspondences. However, the explanation for these various correspondences with the chemical senses may turn out to be qualitatively different, given the presence of identifiable source objects in the case of food aromas/flavours, but not necessarily in the case of basic tastes. While the internalization of the crossmodal statistics of the environment provides one appealing account for the existence of colour-taste correspondences, emotional mediation may also be relevant. Ultimately, while explaining colour-taste correspondences is of both theoretical and historical interest, the growing awareness of the robustness of colour-taste correspondences would currently seem to be of particular relevance to those working in the fields of design and multisensory experiential marketing.

Sonic Seasoning and Other Multisensory Influences on the Coffee Drinking Experience

The coffee drinking experience undoubtedly depends greatly on the quality of the coffee bean and the method of preparation. However, beyond the product-intrinsic qualities of the beverage itself, there are also a host of other product-extrinsic factors that have been shown to influence the coffee-drinking experience. This review summarizes the influence of everything from the multisensory atmosphere through to the sound of coffee preparation, and from the typeface on the coffee packaging through the drinking vessel. Furthermore, the emerging science around sonic seasoning, whereby specific pieces of music or soundscapes, either pre-composed or bespoke, are used to bring out specific aspects in the taste (e.g., sweetness or bitterness) or aroma/flavor (nutty, dark chocolate, dried fruit notes, etc.) of a coffee beverage is also discussed in depth. Relevant related research with other complex drinks such as beer and wine are also mentioned where relevant.

Virtual Terroir and the Premium Coffee Experience

With its origin-centric value proposition, the specialty coffee industry seeks to educate consumers about the value of the origin of coffee and how the relationship with farmers ensures quality and makes coffee a premium product. While the industry has widely used stories and visual cues to communicate this added value, research studying whether and how these efforts influence consumers' experiences is scarce. Through three experiments, we explored the effect of images that evoke the terroir of coffee on the perception of premiumness. Our results revealed that online images that resembled the broad origin of coffee (i.e., a farm) could influence premiumness expectations of coffee (Experiment 1). Similarly, a virtual reality environment that depicted this broad origin (vs. a control but not a city atmosphere) could enhance the perception of coffee premiumness for non-expert consumers (Experiment 2) and the enjoyment of the experience for coffee professionals (Experiment 3). Importantly, we found that congruence between the coffee and the virtual reality (VR) atmospheres mediated how much non-experts enjoyed the experience (Experiment 2). VR atmospheres also influenced expectations of sweetness and acidity for non-experts (Experiment 2). These findings serve as a steppingstone for further exploration of the effects of congruence between visual cues and product/brand attributes on premiumness expectations and perception, and more generally on consumer experience. From a practical standpoint, this study provides insights into key aspects for the development of immersive virtual product experiences.

A Study of Multi-Sensory Experience and Color Recognition in Visual Arts Appreciation of People with Visual Impairment

Visually impaired visitors experience many limitations when visiting museum exhibits, such as a lack of cognitive and sensory access to exhibits or replicas. Contemporary art is evolving in the direction of appreciation beyond simply looking at works, and the development of various sensory technologies has had a great influence on culture and art. Thus, opportunities for people with visual impairments to appreciate visual artworks through various senses such as hearing, touch, and smell are expanding. However, it is uncommon to provide an interactive interface for color recognition, such as applying patterns, sounds, temperature, or scents. This review aims to convey the visual elements of the work to the visually impaired through various sensory elements. In addition, to open a new perspective on appreciation of the works, the technique of expressing the color coded by integrating patterns, temperature, scent, music, and vibration was explored, and future research topics were presented.

Sooo Sweeet! Presence of Long Vowels in Brand Names Lead to Expectations of Sweetness

Throughout the history of languages, poets and writers have used linguistic tools to enhance euphony in their creations. One of the widely used tools to convey melody in any written (or spoken) creative art form is the use of long vowels. This paper examines the linkages between long (vs. short) vowel sounds and taste expectations of sweetness. Across four studies, we demonstrate that people expect products with brand names containing long vowels to taste sweeter than those including short vowel sounds. In studies 1 and 2, we demonstrate this association with the use of self-reported measures, and in studies 3 and 4, we employ indirect measures (implicit taste-shape correspondence and Single Category Implicit Association Test (SC-IAT) paradigm) to show the effect holds at a subconscious level of processing. Previous research in this field has typically linked vowel position (high vs. low or front vs. back) with product or brand attribute expectations. This paper contributes to the growing body of literature in this field by demonstrating the importance of vowel length in sound symbolism, and more precisely, how it pertains to the taste continuum.

Crossmodal correspondences as common ground for joint action

When performing joint actions, people rely on common ground - shared information that provides the required basis for mutual understanding. Common ground can be based on people's interaction history or on knowledge and expectations people share, e.g., because they belong to the same culture or social class. Here, we suggest that people rely on yet another form of common ground, one that originates in their similarities in multisensory processing. Specifically, we focus on 'crossmodal correspondences' - nonarbitrary associations that people make between stimulus features in different sensory modalities, e.g., between stimuli in the auditory and the visual modality such as high-pitched sounds and small objects. Going beyond previous research that focused on investigating crossmodal correspondences in individuals, we propose that people can use these correspondences for communicating and coordinating with others. Initial support for our proposal comes from a communication game played in a public space (an art gallery) by pairs of visitors. We observed that pairs created nonverbal communication systems by spontaneously relying on 'crossmodal common ground'. Based on these results, we conclude that crossmodal correspondences not only occur within individuals but that they can also be actively used in joint action to facilitate the coordination between individuals.

Olfactory-colour crossmodal correspondences in art, science, and design

The last few years have seen a rapid growth of interest amongst researchers in the crossmodal correspondences. One of the correspondences that has long intrigued artists is the putative association between colours and odours. While traditionally conceptualised in terms of synaesthesia, over the last quarter century or so, at least 20 published peer-reviewed articles have assessed the consistent, and non-random, nature of the colours that people intuitively associate with specific (both familiar and unfamiliar) odours in a non-food context. Having demonstrated such consistent mappings amongst the general (i.e. non-synaesthetic) population, researchers have now started to investigate whether they are shared cross-culturally, and to document their developmental acquisition. Over the years, several different explanations have been put forward by researchers for the existence of crossmodal correspondences, including the statistical, semantic, structural, and emotional-mediation accounts. While several of these approaches would appear to have some explanatory validity as far as the odour-colour correspondences are concerned, contemporary researchers have focussed on learned associations as the dominant explanatory framework. The nature of the colour-odour associations that have been reported to date appear to depend on the familiarity of the odour and the ease of source naming, and hence the kind of association/representation that is accessed. While the bidirectionality of odour-colour correspondences has not yet been rigorously assessed, many designers are nevertheless already starting to build on odour-colour crossmodal correspondences in their packaging/labelling/branding work.

Assessing the Role of Emotional Mediation in Explaining Crossmodal Correspondences Involving Musical Stimuli

A wide variety of crossmodal correspondences, defined as the often surprising connections that people appear to experience between simple features, attributes, or dimensions of experience, either physically present or else merely imagined, in different sensory modalities, have been demonstrated in recent years. However, a number of crossmodal correspondences have also been documented between more complex (i.e., multi-component) stimuli, such as, for example, pieces of music and paintings. In this review, the extensive evidence supporting the emotional mediation account of the crossmodal correspondences between musical stimuli (mostly pre-recorded short classical music excerpts) and visual stimuli, including colour patches through to, on occasion, paintings, is critically evaluated. According to the emotional mediation account, it is the emotional associations that people have with stimuli that constitutes one of the fundamental bases on which crossmodal associations are established. Taken together, the literature that has been published to date supports emotional mediation as one of the key factors underlying the crossmodal correspondences involving emotionally-valenced stimuli, both simple and complex.

Wine psychology: basic & applied

Basic cognitive research can help to explain our response to wine, and the myriad factors that affect it. Wine is a complex, culture-laden, multisensory stimulus, and our perception/experience of its properties is influenced by everything from the packaging in which it is presented through the glassware in which it is served and evaluated. A growing body of experiential wine research now demonstrates that a number of contextual factors, including everything from the colour of the ambient lighting through to background music can exert a profound, and in some cases predictable, influence over the tasting experience. Sonic seasoning - that is, the matching of music or soundscapes with specific wines in order to accentuate or draw attention to certain qualities/attributes in the wine, such as sweetness, length, or body, also represents a rapidly growing area of empirical study. While such multisensory, experiential wine research undoubtedly has a number of practical applications, it also provides insights concerning multisensory perception that are relevant to basic scientists. Furthermore, the findings of the wine research are also often relevant to those marketers interested in understanding how the consumers' perception of any other food or beverage product can potentially be modified.

Heaviness-brightness correspondence and stimulus-response compatibility

Cross-sensory correspondences can reflect crosstalk between aligned conceptual feature dimensions, though uncertainty remains regarding the identities of all the dimensions involved. It is unclear, for example, if heaviness contributes to correspondences separately from size. Taking steps to dissociate variations in heaviness from variations in size, the question was asked if a heaviness-brightness correspondence will induce a congruity effect during the speeded brightness classification of simple visual stimuli. Participants classified the stimuli according to whether they were brighter or darker than the mid-gray background against which they appeared. They registered their speeded decisions by manipulating (e.g., tapping) the object they were holding in either their left or right hand (e.g., left for bright, right for dark). With these two otherwise identical objects contrasting in their weight, stimuli were classified more quickly when the relative heaviness of the object needing to be manipulated corresponded with the brightness of the stimulus being classified (e.g., the heavier object for a darker stimulus). This novel congruity effect, in the guise of a stimulus-response (S-R) compatibility effect, was induced when heaviness was isolated as an enduring feature of the object needing to be manipulated. It was also undiminished when participants completed a concurrent verbal memory load task, countering claims that the heaviness-brightness correspondence is verbally mediated. Heaviness, alongside size, appears to contribute to cross-sensory correspondences in its own right and in a manner confirming the far-reaching influence of correspondences, extending here to the fluency with which people communicate simple ideas by manipulating a hand-held object.

Multisensory Flavour Perception: Blending, Mixing, Fusion, and Pairing Within and Between the Senses

This review summarizes the various outcomes that may occur when two or more elements are paired in the context of flavour perception. In the first part, I review the literature concerning what happens when flavours, ingredients, and/or culinary techniques are deliberately combined in a dish, drink, or food product. Sometimes the result is fusion but, if one is not careful, the result can equally well be confusion instead. In fact, blending, mixing, fusion, and flavour pairing all provide relevant examples of how the elements in a carefully-crafted multi-element tasting experience may be combined. While the aim is sometimes to obscure the relative contributions of the various elements to the mix (as in the case of blending), at other times, consumers/tasters are explicitly encouraged to contemplate/perceive the nature of the relationship between the contributing elements instead (e.g., as in the case of flavour pairing). There has been a noticeable surge in both popular and commercial interest in fusion foods and flavour pairing in recent years, and various of the 'rules' that have been put forward to help explain the successful combination of the elements in such food and/or beverage experiences are discussed. In the second part of the review, I examine the pairing of flavour stimuli with music/soundscapes, in the emerging field of 'sonic seasoning'. I suggest that the various perceptual pairing principles/outcomes identified when flavours are paired deliberately can also be meaningfully extended to provide a coherent framework when it comes to categorizing the ways in which what we hear can influence our flavour experiences, both in terms of the sensory-discriminative and hedonic response.

Crossmodal Correspondences in the Sounds of Chinese Instruments

People tend to associate stimuli from different sensory modalities, a phenomenon known as crossmodal correspondences. We conducted two experiments to investigate how Chinese participants associated musical notes produced by four types of Chinese instruments (bowed strings, plucked strings, winds, and percussion instruments) with different colors, taste terms, and fabric textures. Specifically, the participants were asked to select a sound to match each color patch or taste term in Experiment 1 and to match the experience of touching each fabric in Experiment 2. The results demonstrated some associations between pitch and color, taste term, or the smoothness of fabrics. Moreover, certain types of Chinese instruments were preferentially chosen to match some of the colors, taste terms, and the texture of certain fabrics. These findings therefore provided insights about the perception of Chinese music and shed light on how to apply the multisensory features of sounds to enhance the composition, performance, and appreciation of music.

What Does a Horgous Look Like? Nonsense Words Elicit Meaningful Drawings

To what extent do people attribute meanings to "nonsense" words? How general is such attribution of meaning? We used a set of words lacking conventional meanings to elicit drawings of made-up creatures. Separate groups of participants rated the nonsense words and the drawings on several semantic dimensions and selected what name best corresponded to each creature. Despite lacking conventional meanings, "nonsense" words elicited a high level of consistency in the produced drawings. Meaning attributions made to nonsense words corresponded with meaning attributions made by separate people to drawings that were inspired by the name. Naïve participants were able to recover the name that inspired the drawing with greater-than-chance accuracy. These results suggest that people make liberal and consistent use of non-arbitrary relationships between forms and meanings. OPEN RESEARCH BADGES: This article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the Open Science Framework at https://osf.io/8juyc/. Learn more about the Open Practices badges from the Center for Open Science: https://osf.io/tvyxz/wiki.

Sensus Communis: Some Perspectives on the Origins of Non-synchronous Cross-Sensory Associations

Adults readily make associations between stimuli perceived consecutively through different sense modalities, such as shapes and sounds. Researchers have only recently begun to investigate such correspondences in infants but only a handful of studies have focused on infants less than a year old. Are infants able to make cross-sensory correspondences from birth? Do certain correspondences require extensive real-world experience? Some studies have shown that newborns are able to match stimuli perceived in different sense modalities. Yet, the origins and mechanisms underlying these abilities are unclear. The present paper explores these questions and reviews some hypotheses on the emergence and early development of cross-sensory associations and their possible links with language development. Indeed, if infants can perceive cross-sensory correspondences between events that share certain features but are not strictly contingent or co-located, one may posit that they are using a "sixth sense" in Aristotle's sense of the term. And a likely candidate for explaining this mechanism, as Aristotle suggested, is movement.

On the Relationship(s) Between Color and Taste/Flavor

Experimental psychologists, psychophysicists, food/sensory scientists, and marketers have long been interested in, and/or speculated about, what exactly the relationship, if any, might be between color and taste/flavor. While several influential early commentators argued against there being any relationship, a large body of empirical evidence published over the last 80 years or so clearly demonstrates that the hue and saturation, or intensity, of color in food and/or drink often influences multisensory flavor perception. Interestingly, the majority of this research has focused on vision's influence on the tasting experience rather than looking for any effects in the opposite direction. Recently, however, a separate body of research linking color and taste has emerged from the burgeoning literature on the crossmodal correspondences. Such correspondences, or associations, between attributes or dimensions of experience, are thought to be robustly bidirectional. When talking about the relationship between color and taste/flavor, some commentators would appear to assume that these two distinct literatures describe the same underlying empirical phenomenon. That said, a couple of important differences (in terms of the bidirectionality of the effects and their relative vs. absolute nature) are highlighted, meaning that the findings from one domain may not necessarily always be transferable to the other, as is often seemingly assumed.

A Sweet Voice: The Influence of Cross-Modal Correspondences Between Taste and Vocal Pitch on Advertising Effectiveness

We have seen a rapid growth of interest in cross-modal correspondences between sound and taste over recent years. People consistently associate higher-pitched sounds with sweet/sour foods, while lower-pitched sounds tend to be associated with bitter foods. The human voice is key in broadcast advertising, and the role of voice in communication generally is partly characterized by acoustic parameters of pitch. However, it remains unknown whether voice pitch and taste interactively influence consumer behavior. Since consumers prefer congruent sensory information, it is plausible that voice pitch and taste interactively influence consumers’ responses to advertising stimuli. Based on the cross-modal correspondence phenomenon, this study aimed to elucidate the role played by voice pitch–taste correspondences in advertising effectiveness. Participants listened to voiceover advertisements (at a higher or lower pitch than the original narrator’s voice) for three food products with distinct tastes (sweet, sour, and bitter) and rated their buying intention (an indicator of advertising effectiveness). The results show that the participants were likely to exhibit greater buying intention toward both sweet and sour food when they listened to higher-pitched (vs lower-pitched) voiceover advertisements. The influence of a higher pitch on sweet and sour food preferences was observed in only two of the three studies: studies 1 and 2 for sour food, and studies 2 and 3 for sweet food. These findings emphasize the role that voice pitch–taste correspondence plays in preference formation, and advance the applicability of cross-modal correspondences to business.

Taste the Bass: Low Frequencies Increase the Perception of Body and Aromatic Intensity in Red Wine

Associations between heaviness and bass/low-pitched sounds reverberate throughout music, philosophy, literature, and language. Given that recent research into the field of cross-modal correspondences has revealed a number of robust relationships between sound and flavour, this exploratory study was designed to investigate the effects of lower frequency sound (10 Hz to 200 Hz) on the perception of the mouthfeel character of palate weight/body. This is supported by an overview of relevant cross-modal studies and cultural production. Wines were the tastants - a New Zealand Pinot Noir and a Spanish Garnacha - which were tasted in silence and with a 100 Hz (bass) and a higher 1000 Hz sine wave tone. Aromatic intensity was included as an additional character given suggestions that pitch may influence the perception of aromas, which might presumably affect the perception of wine body. Intensity of acidity and liking were also evaluated. The results revealed that the Pinot Noir wine was rated as significantly fuller-bodied when tasted with a bass frequency than in silence or with a higher frequency sound. The low frequency stimulus also resulted in the Garnacha wine being rated as significantly more aromatically intense than when tasted in the presence of the higher frequency auditory stimulus. Acidity was rated considerably higher with the higher frequency in both wines by those with high wine familiarity and the Pinot Noir significantly better liked than the Garnacha. Possible reasons as to why the tones used in this study affected perception of the two wines differently are discussed. Practical application of the findings are also proposed.

Attending to the Chemical Senses

Theorizing around the topic of attention and its role in human information processing largely emerged out of research on the so-called spatial senses: vision, audition, and to a lesser extent, touch. Thus far, the chemical senses have received far less research interest (or should that be attention) from those experimental psychologists and cognitive neuroscientists interested in the topic. Nevertheless, this review highlights the key role that attentional selection also plays in chemosensory information processing and awareness. Indeed, many of the same theoretical approaches/experimental paradigms that were originally developed in the context of the spatial senses, can be (and in some cases already have been) extended to provide a useful framework for thinking about the perception of taste/flavour. Furthermore, a number of those creative individuals interested in modifying the perception of taste/flavour by manipulating product-extrinsic cues (such as, for example, music in the case of sonic seasoning) are increasingly looking to attentional accounts in order to help explain the empirical phenomena that they are starting to uncover. However, separate from its role in explaining sonic seasoning, gaining a better understanding of the role of attentional distraction in modulating our eating/drinking behaviours really ought to be a topic of growing societal concern. This is because distracted diners (e.g., those who eat while watching TV, fiddling with a mobile device or smartphone, or even while driving) consume significantly more than those who mindfully pay attention to the sensations associated with eating and drinking.

On the Relative Nature of (Pitch-Based) Crossmodal Correspondences

This review deals with the question of the relative vs absolute nature of crossmodal correspondences, with a specific focus on those correspondences involving the auditory dimension of pitch. Crossmodal correspondences have been defined as the often-surprising crossmodal associations that people experience between features, attributes, or dimensions of experience in different sensory modalities, when either physically present, or else merely imagined. In the literature, crossmodal correspondences have often been contrasted with synaesthesia in that the former are frequently said to be relative phenomena (e.g., it is the higher-pitched of two sounds that is matched with the smaller of two visual stimuli, say, rather than there being a specific one-to-one crossmodal mapping between a particular pitch of sound and size of object). By contrast, in the case of synaesthesia, the idiosyncratic mapping between inducer and concurrent tends to be absolute (e.g., it is a particular sonic inducer that elicits a specific colour concurrent). However, a closer analysis of the literature soon reveals that the distinction between relative and absolute in the case of crossmodal correspondences may not be as clear-cut as some commentators would have us believe. Furthermore, it is important to note that the relative vs absolute question may receive different answers depending on the particular (class of) correspondence under empirical investigation.