1
|
Abstract
Across the millennia, and across a range of disciplines, there has been a widespread desire to connect, or translate between, the senses in a manner that is meaningful, rather than arbitrary. Early examples were often inspired by the vivid, yet mostly idiosyncratic, crossmodal matches expressed by synaesthetes, often exploited for aesthetic purposes by writers, artists, and composers. A separate approach comes from those academic commentators who have attempted to translate between structurally similar dimensions of perceptual experience (such as pitch and colour). However, neither approach has succeeded in delivering consensually agreed crossmodal matches. As such, an alternative approach to sensory translation is needed. In this narrative historical review, focusing on the translation between audition and vision, we attempt to shed light on the topic by addressing the following three questions: (1) How is the topic of sensory translation related to synaesthesia, multisensory integration, and crossmodal associations? (2) Are there common processing mechanisms across the senses that can help to guarantee the success of sensory translation, or, rather, is mapping among the senses mediated by allegedly universal (e.g., amodal) stimulus dimensions? (3) Is the term 'translation' in the context of cross-sensory mappings used metaphorically or literally? Given the general mechanisms and concepts discussed throughout the review, the answers we come to regarding the nature of audio-visual translation are likely to apply to the translation between other perhaps less-frequently studied modality pairings as well.
Collapse
Affiliation(s)
- Charles Spence
- Crossmodal Research Laboratory, University of Oxford, Oxford, UK.
- Department of Experimental Psychology, New Radcliffe House, University of Oxford, Oxford, OX2 6BW, UK.
| | - Nicola Di Stefano
- Institute of Cognitive Sciences and Technologies, National Research Council of Italy (CNR), Rome, Italy
| |
Collapse
|
2
|
Crichton-Fock A, Spence C, Pettersson N. Using crossmodal correspondences as a tool in wine communication. Front Psychol 2023; 14:1190364. [PMID: 37583598 PMCID: PMC10424788 DOI: 10.3389/fpsyg.2023.1190364] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 07/06/2023] [Indexed: 08/17/2023] Open
Abstract
Introduction This research investigates consumer acceptance of alternative methods for communicating information about wine, focusing on the alignment between sensory attributes and consumer expectations. Methods A survey was administered to wine enthusiasts to assess their attitudes toward crossmodal communication. Results The findings reveal significant associations between consumer behaviors and acceptance of alternative communication methods, highlighting the emerging field of crossmodal correspondences. Discussion These results suggest that leveraging crossmodal correspondences can enhance the match between a product's sensory qualities and consumer expectations, potentially reducing wine wastage resulting from unmet consumer preferences. These findings have implications for improving communication strategies in the wine industry and enhancing consumer experiences.
Collapse
Affiliation(s)
- Anders Crichton-Fock
- School of Hospitality, Culinary Arts & Meal Science, Örebro University, Örebro, Sweden
| | - Charles Spence
- Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| | | |
Collapse
|
3
|
Pinardi M, Di Stefano N, Di Pino G, Spence C. Exploring crossmodal correspondences for future research in human movement augmentation. Front Psychol 2023; 14:1190103. [PMID: 37397340 PMCID: PMC10308310 DOI: 10.3389/fpsyg.2023.1190103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 05/30/2023] [Indexed: 07/04/2023] Open
Abstract
"Crossmodal correspondences" are the consistent mappings between perceptual dimensions or stimuli from different sensory domains, which have been widely observed in the general population and investigated by experimental psychologists in recent years. At the same time, the emerging field of human movement augmentation (i.e., the enhancement of an individual's motor abilities by means of artificial devices) has been struggling with the question of how to relay supplementary information concerning the state of the artificial device and its interaction with the environment to the user, which may help the latter to control the device more effectively. To date, this challenge has not been explicitly addressed by capitalizing on our emerging knowledge concerning crossmodal correspondences, despite these being tightly related to multisensory integration. In this perspective paper, we introduce some of the latest research findings on the crossmodal correspondences and their potential role in human augmentation. We then consider three ways in which the former might impact the latter, and the feasibility of this process. First, crossmodal correspondences, given the documented effect on attentional processing, might facilitate the integration of device status information (e.g., concerning position) coming from different sensory modalities (e.g., haptic and visual), thus increasing their usefulness for motor control and embodiment. Second, by capitalizing on their widespread and seemingly spontaneous nature, crossmodal correspondences might be exploited to reduce the cognitive burden caused by additional sensory inputs and the time required for the human brain to adapt the representation of the body to the presence of the artificial device. Third, to accomplish the first two points, the benefits of crossmodal correspondences should be maintained even after sensory substitution, a strategy commonly used when implementing supplementary feedback.
Collapse
Affiliation(s)
- Mattia Pinardi
- NeXT Lab, Neurophysiology and Neuroengineering of Human-Technology Interaction Research Unit, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Nicola Di Stefano
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Giovanni Di Pino
- NeXT Lab, Neurophysiology and Neuroengineering of Human-Technology Interaction Research Unit, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Charles Spence
- Crossmodal Research Laboratory, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
4
|
Swahn J, Nilsen A. ‘Sounds salty!’ How a soundtrack affects the liking and perception of the salty balance in bread. Int J Gastron Food Sci 2023. [DOI: 10.1016/j.ijgfs.2023.100718] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/03/2023]
|
5
|
Spence C. Gastrophysics: Getting creative with pairing flavours. Int J Gastron Food Sci 2022. [DOI: 10.1016/j.ijgfs.2021.100433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
6
|
Spence C. Sonic Seasoning and Other Multisensory Influences on the Coffee Drinking Experience. FRONTIERS IN COMPUTER SCIENCE 2021. [DOI: 10.3389/fcomp.2021.644054] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023] Open
Abstract
The coffee drinking experience undoubtedly depends greatly on the quality of the coffee bean and the method of preparation. However, beyond the product-intrinsic qualities of the beverage itself, there are also a host of other product-extrinsic factors that have been shown to influence the coffee-drinking experience. This review summarizes the influence of everything from the multisensory atmosphere through to the sound of coffee preparation, and from the typeface on the coffee packaging through the drinking vessel. Furthermore, the emerging science around sonic seasoning, whereby specific pieces of music or soundscapes, either pre-composed or bespoke, are used to bring out specific aspects in the taste (e.g., sweetness or bitterness) or aroma/flavor (nutty, dark chocolate, dried fruit notes, etc.) of a coffee beverage is also discussed in depth. Relevant related research with other complex drinks such as beer and wine are also mentioned where relevant.
Collapse
|
7
|
Abstract
Basic cognitive research can help to explain our response to wine, and the myriad factors that affect it. Wine is a complex, culture-laden, multisensory stimulus, and our perception/experience of its properties is influenced by everything from the packaging in which it is presented through the glassware in which it is served and evaluated. A growing body of experiential wine research now demonstrates that a number of contextual factors, including everything from the colour of the ambient lighting through to background music can exert a profound, and in some cases predictable, influence over the tasting experience. Sonic seasoning - that is, the matching of music or soundscapes with specific wines in order to accentuate or draw attention to certain qualities/attributes in the wine, such as sweetness, length, or body, also represents a rapidly growing area of empirical study. While such multisensory, experiential wine research undoubtedly has a number of practical applications, it also provides insights concerning multisensory perception that are relevant to basic scientists. Furthermore, the findings of the wine research are also often relevant to those marketers interested in understanding how the consumers' perception of any other food or beverage product can potentially be modified.
Collapse
Affiliation(s)
- Charles Spence
- Department of Experimental Psychology, Crossmodal Research Laboratory, Anna Watts Building, University of Oxford, Oxford, OX2 6GG, UK.
| |
Collapse
|
8
|
Spence C. Multisensory Flavour Perception: Blending, Mixing, Fusion, and Pairing Within and Between the Senses. Foods 2020; 9:E407. [PMID: 32244690 PMCID: PMC7230593 DOI: 10.3390/foods9040407] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2020] [Revised: 03/21/2020] [Accepted: 03/21/2020] [Indexed: 11/16/2022] Open
Abstract
This review summarizes the various outcomes that may occur when two or more elements are paired in the context of flavour perception. In the first part, I review the literature concerning what happens when flavours, ingredients, and/or culinary techniques are deliberately combined in a dish, drink, or food product. Sometimes the result is fusion but, if one is not careful, the result can equally well be confusion instead. In fact, blending, mixing, fusion, and flavour pairing all provide relevant examples of how the elements in a carefully-crafted multi-element tasting experience may be combined. While the aim is sometimes to obscure the relative contributions of the various elements to the mix (as in the case of blending), at other times, consumers/tasters are explicitly encouraged to contemplate/perceive the nature of the relationship between the contributing elements instead (e.g., as in the case of flavour pairing). There has been a noticeable surge in both popular and commercial interest in fusion foods and flavour pairing in recent years, and various of the 'rules' that have been put forward to help explain the successful combination of the elements in such food and/or beverage experiences are discussed. In the second part of the review, I examine the pairing of flavour stimuli with music/soundscapes, in the emerging field of 'sonic seasoning'. I suggest that the various perceptual pairing principles/outcomes identified when flavours are paired deliberately can also be meaningfully extended to provide a coherent framework when it comes to categorizing the ways in which what we hear can influence our flavour experiences, both in terms of the sensory-discriminative and hedonic response.
Collapse
Affiliation(s)
- Charles Spence
- Crossmodal Research Laboratory, Oxford University, Oxford OX2 6GG, UK
| |
Collapse
|
9
|
Spence C. Food and beverage flavour pairing: A critical review of the literature. Food Res Int 2020; 133:109124. [PMID: 32466920 DOI: 10.1016/j.foodres.2020.109124] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2019] [Revised: 02/22/2020] [Accepted: 02/23/2020] [Indexed: 01/13/2023]
Abstract
The recent explosion of interest in the topic of flavour pairing has been driven, at least in part, by the now-discredited food-pairing hypothesis, along with the emergence of the new field of computational gastronomy. Many chefs, sommeliers, mixologists, and drinks brands, not to mention a few food brands, have become increasingly interested in moving the discussions that they have with their consumers beyond the traditional focus solely on food and wine pairings. Here, two key approaches to pairing that might help to explain/justify those food and beverage combinations that the consumer is likely to appreciate are outlined. Historically-speaking, many conventional pairings emerged naturally from cultural/geographical matches, presumably internalized as semantic knowledge amongst consumers. In this review, such conventional pairings are framed as but one example of a cognitive/intellectual food-beverage strategy. The alternative approach to pairing that has become increasingly popular in recent years involves experts/commentators making recommendations based on the perceptual relationship, or interaction, between the component stimuli, be it one of perceived similarity, contrast, harmony, emergence, or modulation (either suppression or enhancement). Physicochemical accounts of pairing, based on the presence of shared flavour molecules (e.g., aromatic volatiles) in the to-be-combined flavours or ingredients, have also gained in popularity. Here, though, the latter approach is framed as an ultimately unsuccessful attempt to predict matches based on perceived similarity. This review summarizes the available evidence concerning food-beverage pairing and proposes a new dichotomy between intellectual/cognitive and perceptual pairing principles in the case of food-beverage matching.
Collapse
|
10
|
Hauck P, Hecht H. Having a Drink with Tchaikovsky: The Crossmodal Influence of Background Music on the Taste of Beverages. Multisens Res 2019; 32:1-24. [PMID: 31059491 DOI: 10.1163/22134808-20181321] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 06/25/2018] [Indexed: 11/19/2022]
Abstract
Previous research has shown that auditory cues can influence the flavor of food and drink. For instance, wine tastes better when preferred music is played. We have investigated whether a music background can modify judgments of the specific flavor pattern of a beverage, as opposed to mere preference. This was indeed the case. We explored the nature of this crosstalk between auditory and gustatory perception, and hypothesized that the 'flavor' of the background music carries over to the perceived flavor (i.e., descriptive and evaluative aspects) of beverages. First, we collected ratings of the subjective flavor of different music pieces. Then we used a between-subjects design to cross the music backgrounds with taste evaluations of several beverages. Participants tasted four different samples of beverages under two contrasting audio conditions and rated their taste experiences. The emotional flavor of the music had the hypothesized effects on the flavor of the beverages. We also hypothesized that such an effect would be stronger for music novices than for music experts, and weaker for aqueous solutions than for wines. However, neither music expertise nor liquid type produced additional effects. We discuss implications of this audio-gustatory interaction.
Collapse
Affiliation(s)
- Pia Hauck
- Department of Psychology, General Experimental Psychology, Johannes Gutenberg-Universität Mainz, Wallstrasse 3, 55122 Mainz, Germany
| | - Heiko Hecht
- Department of Psychology, General Experimental Psychology, Johannes Gutenberg-Universität Mainz, Wallstrasse 3, 55122 Mainz, Germany
| |
Collapse
|
11
|
|
12
|
Wang QJ, Spence C. Assessing the influence of music on wine perception among wine professionals. Food Sci Nutr 2017; 6:295-301. [PMID: 29564095 PMCID: PMC5849903 DOI: 10.1002/fsn3.554] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2017] [Revised: 10/26/2017] [Accepted: 10/31/2017] [Indexed: 11/23/2022] Open
Abstract
Several recent studies have demonstrated that music can significantly influence the eating/drinking experience. It is not clear, however, whether this influence would be moderated by the expertise of the taster. In the experiments reported here, we tested a large group (N = 154) of very experienced wine tasters—the majority of whom were professionals working in the wine business—at a winemaking conference. The first study assessed the impact of putatively “sweet” and “sour” soundtracks on taste evaluation, whereas the second study assessed more subtle wine‐specific terminology such as length, balance, and body. The results revealed that the effect of music on wine perception can indeed be demonstrated in wine experts. Moreover, the amount of wine tasting experience, as measured in years, did not moderate the influence of music on sensory and hedonic wine evaluation. This result suggests that the aforementioned auditory modulation of drinking experience is not influenced by the increased analytical abilities afforded by traditional wine tasting expertise.
Collapse
Affiliation(s)
- Qian Janice Wang
- Crossmodal Research Laboratory Department of Experimental Psychology Oxford University Oxford UK
| | - Charles Spence
- Crossmodal Research Laboratory Department of Experimental Psychology Oxford University Oxford UK
| |
Collapse
|
13
|
Assessing the Role of Emotional Associations in Mediating Crossmodal Correspondences between Classical Music and Red Wine. BEVERAGES 2017. [DOI: 10.3390/beverages3010001] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
14
|
|
15
|
Spence C, Wang QJ. Wine and music (III): so what if music influences the taste of the wine? ACTA ACUST UNITED AC 2015. [DOI: 10.1186/s13411-015-0046-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|