Augmented reality flavor: cross-modal mapping across gustation, olfaction, and vision

Crossmodal correspondences between basic tastes and visual design features: A narrative historical review

People tend to associate abstract visual features with basic taste qualities. This narrative historical review critically evaluates the literature on these associations, often referred to as crossmodal correspondences, between basic tastes and visual design features such as color hue and shape curvilinearity. The patterns, discrepancies, and evolution in the development of the research are highlighted while the mappings that have been reported to date are summarized. The review also reflects on issues of cross-cultural validity and deviations in the matching patterns that are observed when correspondences are assessed with actual tastants versus with verbal stimuli. The various theories that have been proposed to account for different classes of crossmodal correspondence are discussed, among which the statistical and affective (or emotional-mediation) accounts currently appear most promising. Several critical research questions for the future are presented to address the gaps that have been identified in the literature and help validate the popular theories on the origin and operations of visual-taste correspondences.

Evaluation of Realistic Sensation Using Biological Reaction Measurements for Food Videos Presented by KMMD

We have prototyped a KANSEI multimedia display (KMMD) that is able to release scent through the screen in order to realize collaboration between images and scents. Two types of “sukiyaki” food videos were presented to subjects using this device, and a method for objectively evaluating the realistic sensation of the food videos was examined using biological reaction measurements. The sukiyaki scent was added to one type of video to improve appetite. Viewers’ saliva flow rate, line of sight, pupil diameter, autonomic nerve activity, and cerebral blood flow were measured at the same time, and changes in these measured values were analyzed. As a result, the scent was effective in improving the sensation, as if the food was present in front of the eyes and increasing the saliva flow rate. Additionally, in a realistic scene, it was found that the line of sight follows the performer’s eating behavior as if the viewers themselves are eating. The sympathetic nervous system temporarily increases, mydriasis occurs, and the frontal lobe is activated. Furthermore, the possibility of objective evaluation of realistic sensations was demonstrated by the correlation between appetite, accompanied by salivary sensation, and the biological reaction measurement results.