Using implicit rather than explicit measures of emotions

An advance in novel intelligent sensory technologies: From an implicit-tracking perspective of food perception

Food sensory evaluation mainly includes explicit and implicit measurement methods. Implicit measures of consumer perception are gaining significant attention in food sensory and consumer science as they provide effective, subconscious, objective analysis. A wide range of advanced technologies are now available for analyzing physiological and psychological responses, including facial analysis technology, neuroimaging technology, autonomic nervous system technology, and behavioral pattern measurement. However, researchers in the food field often lack systematic knowledge of these multidisciplinary technologies and struggle with interpreting their results. In order to bridge this gap, this review systematically describes the principles and highlights the applications in food sensory and consumer science of facial analysis technologies such as eye tracking, facial electromyography, and automatic facial expression analysis, as well as neuroimaging technologies like electroencephalography, magnetoencephalography, functional magnetic resonance imaging, and functional near-infrared spectroscopy. Furthermore, we critically compare and discuss these advanced implicit techniques in the context of food sensory research and then accordingly propose prospects. Ultimately, we conclude that implicit measures should be complemented by traditional explicit measures to capture responses beyond preference. Facial analysis technologies offer a more objective reflection of sensory perception and attitudes toward food, whereas neuroimaging techniques provide valuable insight into the implicit physiological responses during food consumption. To enhance the interpretability and generalizability of implicit measurement results, further sensory studies are needed. Looking ahead, the combination of different methodological techniques in real-life situations holds promise for consumer sensory science in the field of food research.

Sensory and affective response to chocolate differing in cocoa content: A TDS and facial electromyography approach

Existing research has offered insight into facial activities and their associations with hedonic liking during the consumption of basic food samples and suggests facial changes during consumption are linked to the hedonic evaluation of tastes and, thus related to the taster's perception rather than the tastes themselves. This study tests whether, during the consumption of commercially available dark chocolate, a complex food product, which can be high in bitterness but expectedly so, how facial activities are linked to the bitterness levels and the hedonic liking of the samples. To do this we carried out two studies with untrained consumers, the first of which captured temporally dynamic sensory perception during the consumption of dark chocolate samples of 36% and 85% cocoa content, using the Temporal Dominance of Sensations (TDS) approach. The second study captured facial EMG over the corrugator and zygomaticus muscles during the consumption of dark chocolate samples (36%, 70%, and 85% cocoa). Specifically, the aim of this research was to investigate whether corrugator activity had a greater association with bitterness perception, linked to cocoa, or hedonic evaluation. Capturing the dynamic sensory profile of chocolate samples allowed an investigation into the time points most evident of sensory variation related to the bitterness and sweetness of the taste, allowing insight into whether facial activities also deviated during this time. These data offer evidence to suggest that corrugator was associated with hedonic evaluation during consumption of the samples, with the most liked samples (being those with 70% and 36% cocoa) eliciting similar corrugator activities and less activity than the least liked 85% cocoa content sample; however, there was also evidence to suggest a significant variation in participants' corrugator activity during the period of oral processing when bitterness was most evident in the 85% cocoa sample and sweetness was most evident in the 36% cocoa sample (i.e., the time when bitterness and sweetness were most divergent) Further investigation showed a variation in facial activities elicited during consumption of the 36% cocoa sample based on whether individuals were part of the group who favoured the 85% cocoa sample or the group favouring the 36% cocoa sample. The findings, therefore, suggest facial EMG, specifically over the corrugator, appears to be related to the hedonic evaluation of a complex food product and not the taste itself. Furthermore, being aware of the time points where sensory variations are most apparent between samples can allow for targeted investigation into facial EMG and its ability to distinguish food samples.

Predicting Perceived Hedonic Ratings through Facial Expressions of Different Drinks

Previous studies have established the utility of facial expressions as an objective assessment approach for determining the hedonics (overall pleasure) of food and beverages. This study endeavors to validate the conclusions drawn from preceding research, illustrating that facial expressions prompted by tastants possess the capacity to forecast the perceived hedonic ratings of these tastants. Facial expressions of 29 female participants, aged 18-55 years, were captured using a digital camera during their consumption of diverse concentrations of solutions representative of five basic tastes. Employing the widely employed facial expression analysis application FaceReader, the facial expressions were meticulously assessed, identifying seven emotions (surprise, happiness, scare, neutral, disgust, sadness, and anger) characterized by scores ranging from 0 to 1-a numerical manifestation of emotional intensity. Simultaneously, participants rated the hedonics of each solution, utilizing a scale spanning from -5 (extremely unpleasant) to +5 (extremely pleasant). Employing a multiple linear regression analysis, a predictive model for perceived hedonic ratings was devised. The model's efficacy was scrutinized by assessing emotion scores from 11 additional taste solutions, sampled from 20 other participants. The anticipated hedonic ratings demonstrated robust alignment and agreement with the observed ratings, underpinning the validity of earlier findings even when incorporating diverse software and taste stimuli across a varied participant base. We discuss some limitations and practical implications of our technique in predicting food and beverage hedonics using facial expressions.

The Application of Biometric Approaches in Agri-Food Marketing: A Systematic Literature Review

A challenge in social marketing studies is the cognitive biases in consumers' conscious and self-reported responses. To help address this concern, biometric techniques have been developed to obtain data from consumers' implicit and non-verbal responses. A systematic literature review was conducted to explore biometric applications' role in agri-food marketing to provide an integrated overview of this topic. A total of 55 original research articles and four review articles were identified, classified, and reviewed. It was found that there is a steady growth in the number of studies applying biometric approaches, with eye-tracking being the dominant method used to investigate consumers' perceptions in the last decade. Most of the studies reviewed were conducted in Europe or the USA. Other biometric techniques used included facial expressions, heart rate, body temperature, and skin conductance. A wide range of scenarios concerning consumers' purchase and consumption behaviour for agri-food products have been investigated using biometric-based techniques, indicating their broad applicability. Our findings suggest that biometric techniques are expanding for researchers in agri-food marketing, benefiting both academia and industry.

Consumer Studies: Beyond Acceptability—A Case Study with Beer

Beer is one of the most consumed alcoholic beverages in the world; its consumption and preference are evolving from traditional industrial beers of low complexity to novel craft beers with diverse flavour profiles. In such a competitive industry and considering the complexity of consumer behaviour, improvement and innovation become necessary. Consequently, consumer science, which is responsible for identifying the motivation behind customer preferences through their attitudes, perception and behaviour, has implemented strategies ranging from simple hedonic measurements to several innovative and emerging methodologies for a deeper understanding of the variables that affect the product experience: sensory, affective and cognitive. In this context, we offer a review inspired by previous research that explores some of the quantitative and qualitative methods used in consumer studies related to beer consumption, ranging from traditional approaches (acceptability, purchase intention, preference, etc.) to techniques that go beyond acceptability and allow a different understanding of aspects of consumer perception and behaviour (segmentation, expectations, emotions, representation, etc.). Also, innovative applications (contexts, immersive technologies and virtual reality, implicit measures, etc.) and current trends related to consumer science (Internet, social media, pairing, product experience, etc.) are addressed.

Exploratory Research on Sweetness Perception: Decision Trees to Study Electroencephalographic Data and Its Relationship with the Explicit Response to Sweet Odor, Taste, and Flavor

Using implicit responses to determine consumers' response to different stimuli is becoming a popular approach, but research is still needed to understand the outputs of the different technologies used to collect data. During the present research, electroencephalography (EEG) responses and self-reported liking and emotions were collected on different stimuli (odor, taste, flavor samples) to better understand sweetness perception. Artificial intelligence analytics were used to classify the implicit responses, identifying decision trees to discriminate the stimuli by activated sensory system (odor/taste/flavor) and by nature of the stimuli ('sweet' vs. 'non-sweet' odors; 'sweet-taste', 'sweet-flavor', and 'non-sweet flavor'; and 'sweet stimuli' vs. 'non-sweet stimuli'). Significant differences were found among self-reported-liking of the stimuli and the emotions elicited by the stimuli, but no clear relationship was identified between explicit and implicit data. The present research sums interesting data for the EEG-linked research as well as for EEG data analysis, although much is still unknown about how to properly exploit implicit measurement technologies and their data.

Co-Creation with Consumers for Packaging Design Validated through Implicit and Explicit Methods: Exploratory Effect of Visual and Textual Attributes

Packaging is no longer a mere structural element that only aims to preserve foods, but it is also a powerful marketing tool able to affect product perception, purchase decision and consumers’ food choices. Incorporating consumers’ voices into packaging design through co-creation could maximise its impact on the market. The main goal of this exploratory study was to test the usefulness of co-creation with consumers for packaging design. For that purpose, a survey with 200 Spanish participants was conducted to find out which of the presented visual and textual packaging attributes were the most appropriate. A validation study with 40 participants using implicit (eye tracker, galvanic skin response and automatic facial expression analysis) and explicit measurements was used to test the packaging co-created by consumers against some of its possible competitors in the market. The co-creation process with consumers allowed for the identification of the visual and textual attributes, among the available options, that best fit their preferences, whereas the validation process confirmed that the packaging design co-created by consumers was equally or even preferred over the competitors. The information gathered might help designers and marketers to guide the packaging design for fish products in the Spanish market.

New Consumer Research Technology for Food Behaviour: Overview and Validity

BACKGROUND

the last decade has witnessed an explosion of new consumer behaviour research technology, and new methods are published almost monthly. To what extent are these methods applicable in the specific area of food consumer science, and if they are, are they any good?

METHODS

in this paper, we attempt to give an overview of the developments in this area. We distinguish between ('input') methods needed to shape the measurement context a consumer is brought in, e.g., by means of 'immersive' methods, and ('output') methods that perform measurements proper. Concerning the latter, we distinguish between methods focusing on neuro-science, on psychology, and on behaviour. In addition, we suggest a way to assess the validity of the methods, based on psychological theory, concerning biases resulting from consumer awareness of a measurement situation. The methods are evaluated on three summarising validity criteria; conclusions: the conclusion is that behavioural measures generally appear more valid than psychological or neuro-scientific methods. The main conclusion is that validity of a method should never be taken for granted, and it should be always be assessed in the context of the research question.

Comparison of Self-Report Questionnaire and Eye Tracking Method in the Visual Preference Study of a Youth–Beverage Model

This study compares the characteristics of a self-report questionnaire (SRQ) and eye tracking (ET) based on a simple human–beverage visual cognition model. The young participants were mainly defined by their gender and body mass index (BMI). The beverage samples consisted of milk, coffee, cup, and coaster. SRQs allow the participants to clearly express their overall cognition of the samples in the form of vocabulary, while ET captures their hidden thinking process. The analysis, using a random forest (RF) classifier, found that participant parameters (gender and BMI) played a more important role for SRQ, while ET was related to beverage parameters (color and shape). This work reiterates that these two methods have their advantages and complement each other in food sensory analysis.

Nutraceutical Chewing Candy Formulations Based on Acetic, Alcoholic, and Lactofermented Apple Juice Products

The aim of this study was to develop nutraceutical chewing candy (NCC) formulations based on acetic, alcoholic, and lactofermented apple juice (AJ) products. In addition, different texture-forming (gelatin, pectin) and sweetening (stevia, xylitol) agents were tested. To implement the aim of this study, combinations based on AJ, prepared from fresh and frozen apples, apple cider (C) samples (No.1, No.2, No.3, and No.4), and apple vinegar (V) were used. First, the most appropriate combination was selected by evaluating overall acceptability (OA) and emotions induced for consumers (EIC). In addition, the volatile compound (VC) profile, and physicochemical and antimicrobial characteristics of the developed combinations were analyzed. For AJ fermentation, lactic acid bacteria (LAB) strains possessing antimicrobial properties (LUHS122—L. plantarum and LUHS210—L. casei) were used. AJ prepared from frozen apples had 11.1% higher OA and 45.9%, 50.4%, and 33.3% higher fructose, glucose, and saccharose concentrations, respectively. All the tested C samples inhibited Bacillus subtilis and had an average OA of 6.6 points. Very strong positive correlations were found between AJ and C OA and the emotion ‘happy’; comparing lactofermented AJ, the highest OA was obtained for AJ fermented for 48 h with LUHS122, and a moderate positive correlation was found between AJ OA and the emotion ‘happy’ (r = 0.7617). This sample also showed the highest viable LAB count (7.59 log₁₀ CFU mL⁻¹) and the broadest spectrum of pathogen inhibition (inhibited 6 out of 10 tested pathogens). Further, acetic, alcoholic, and lactofermented AJ product combinations were tested. For the preparation of NCC, the combination consisting of 50 mL of AJ fermented with LUHS122 for 48 h + 50 mL C-No.3 + 2 mL V was selected because it showed the highest OA, induced a high intensity of the emotion ‘happy’ for the judges, and inhibited 8 out of 10 tested pathogens. Finally, the OA of the prepared NCC was, on average, 9.03 points. The combination of acetic, alcoholic, and lactofermented AJ products leads to the formation of a specific VC profile and increases the OA and antimicrobial activity of the products which could be successfully applied in the food and nutraceutical industries.

Reading Food Experiences from the Face: Effects of Familiarity and Branding of Soy Sauce on Facial Expressions and Video-Based RPPG Heart Rate

Food experiences are not only driven by the food's intrinsic properties, such as its taste, texture, and aroma, but also by extrinsic properties such as visual brand information and the consumers' previous experiences with the foods. Recent developments in automated facial expression analysis and heart rate detection based on skin color changes (remote photoplethysmography or RPPG) allow for the monitoring of food experiences based on video images of the face. RPPG offers the possibility of large-scale non-laboratory and web-based testing of food products. In this study, results from the video-based analysis were compared to the more conventional tests (scores of valence and arousal using Emojis and photoplethysmography heart rate (PPG)). Forty participants with varying degrees of familiarity with soy sauce were presented with samples of rice and three commercial soy sauces with and without brand information. The results showed that (1) liking and arousal were affected primarily by the specific tastes, but not by branding and familiarity. In contrast, facial expressions were affected by branding and familiarity, and to a lesser degree by specific tastes. (2) RPPG heart rate and PPG both showed effects of branding and familiarity. However, RPPG heart rate needs further development because it underestimated the heart rate compared to PPG and was less sensitive to changes over time and with activity (viewing of brand information and tasting). In conclusion, this study suggests that recording of facial expressions and heart rates may no longer be limited to laboratories but can be done remotely using video images, which offers opportunities for large-scale testing in consumer science.

Effect of Branding and Familiarity of Soy Sauces on Valence and Arousal as Determined by Facial Expressions, Physiological Measures, Emojis, and Ratings

Food experiences can be summarized along two main dimensions: valence and arousal, which can be measured explicitly with subjective ratings or implicitly with physiological and behavioral measures. Food experiences are not only driven by the food's intrinsic properties, such as its taste, texture, and aroma, but also by extrinsic properties such as brand information and the consumers' previous experiences with the foods. In this study, valence and arousal to intrinsic and extrinsic properties of soy sauce were measured in consumers that varied in their previous experience with soy sauce, using a combination of explicit (scores and emojis), implicit (heart rate and skin conductance), and behavioral measures (facial expressions). Forty participants, high- and low-frequency users, were presented with samples of rice and three commercial soy sauces without and with brand information that either matched or non-matched the taste of the soy sauce. In general, skin conductance and facial expressions showed relatively low arousal during exposure to the brand name and again lowest arousal during tasting. Heart rate was lowest during exposure to the brand name and increased during tasting probably resulting from the motor activity during chewing. Furthermore, the results showed that explicit liking and arousal scores were primarily affected by the taste of the specific soy sauce and by the participants' previous experience with soy sauces. These scores were not affected by branding information. In contrast, facial expressions, skin conductance, and heart rate were primarily affected by (1) the participants' level of experience with soy sauce, (2) whether or not branding information was provided, and (3) whether or not the branding information matched with the taste. In conclusion, this study suggests that liking scores may be most sensitive to the food's intrinsic taste properties, whereas implicit measures and facial expressions may be most sensitive to extrinsic properties such as brand information. All measures were affected by the consumers' previous food experiences.