Autonomic nervous system responses on and facial expressions to the sight, smell, and taste of liked and disliked foods

Development of a Biosensory Computer Application to Assess Physiological and Emotional Responses from Sensory Panelists

In sensory evaluation, there have been many attempts to obtain responses from the autonomic nervous system (ANS) by analyzing heart rate, body temperature, and facial expressions. However, the methods involved tend to be intrusive, which interfere with the consumers’ responses as they are more aware of the measurements. Furthermore, the existing methods to measure different ANS responses are not synchronized among them as they are measured independently. This paper discusses the development of an integrated camera system paired with an Android PC application to assess sensory evaluation and biometric responses simultaneously in the Cloud, such as heart rate, blood pressure, facial expressions, and skin-temperature changes using video and thermal images acquired by the integrated system and analyzed through computer vision algorithms written in Matlab^®, and FaceReader^TM. All results can be analyzed through customized codes for multivariate data analysis, based on principal component analysis and cluster analysis. Data collected can be also used for machine-learning modeling based on biometrics as inputs and self-reported data as targets. Based on previous studies using this integrated camera and analysis system, it has shown to be a reliable, accurate, and convenient technique to complement the traditional sensory analysis of both food and nonfood products to obtain more information from consumers and/or trained panelists.

Personality traits affect the influences of intensity perception and emotional responses on hedonic rating and preference rank toward basic taste solutions

This study aimed at determining, based on independent predictors of taste intensity and emotional response, whether individual personality traits could affect prediction models of overall liking and preference rank toward basic taste solutions. Sixty-seven participants rated taste intensities (TI) of four basic-taste solutions at both low and high concentrations, and of plain water. Emotional responses toward each sample were measured using a self-reported emotion questionnaire (SE), facial expressions (FE), and/or autonomic nervous system responses (ANS). Participants rated overall liking of the samples and ranked their preferences. Based on the results of a hierarchical cluster analysis of five personality traits measured using the Big Five Inventory, participants were classified into two clusters: cluster N (high neuroticism) and cluster E (high extraversion). Results showed that the SE measure for both clusters N and E was better than the TI, FE, and ANS measures in explaining variances of overall liking or preference rank. A measurement of effect size found that using FE and/or taste intensity measures, along with self-reported emotion measure, could enhance model predictability of overall liking or preference rank toward taste samples for cluster N, while the contribution to the prediction model for cluster E was minimal. ANS measures showed little contribution to the prediction model of overall liking for either cluster. In conclusion, this study shows that personality traits, in particular traits of extraversion and neuroticism, affect not only optimum measures of emotional responses, but also modulate predicting overall liking and preference rank toward basic taste solutions.

Effects of emotional responses to certain foods on the prediction of consumer acceptance

This research was focussed on the development of a methodology for the recognition of consumer preferences based on a combination of emotional, behavioural, and sensory trait information. A model of the impact of customers' sensory experiences on their attitudes towards food products and their behavioural intentions was analysed. Sensory and emotional analyses were used to describe five kinds of bread (wheat, rye, corn, wholemeal, and multigrain) and two types of chocolate (dark and milk). Acceptability and emotional response were rated by a consumer panel (n = 109) drawn from 21 to 24 age segments using the hedonic scores and a FaceReader software which detects six basic emotions (happy, sad, angry, disgusted, scared, surprised) and a neutral state were applied. For the products tested, expressions of happiness, anger, and sadness for each product were relatively high compared to the others, with the neutral state being the main expression. The chocolate products elicited the highest intensity of happiness, reducing the level of the neutral state. A different tendency was obtained during the testing of bakery goods: higher expression of the neutral state and sadness and low expression of happiness. The emotional/sensory experience model and consumer behavioural patterns comprise the method for the differentiation of products, which could be useful in the food industry as well as for the development of new methodologies for the prediction of changes in human emotional response to food related to psychological disorders.

Methods for Evaluating Emotions Evoked by Food Experiences: A Literature Review

Besides sensory characteristics of food, food-evoked emotion is a crucial factor in predicting consumer's food preference and therefore in developing new products. Many measures have been developed to assess food-evoked emotions. The aim of this literature review is (i) to give an exhaustive overview of measures used in current research and (ii) to categorize these methods along measurement level (physiological, behavioral, and cognitive) and emotional processing level (unconscious sensory, perceptual/early cognitive, and conscious/decision making) level. This 3 × 3 categorization may help researchers to compile a set of complementary measures (“toolbox”) for their studies. We included 101 peer-reviewed articles that evaluate consumer's emotions and were published between 1997 and 2016, providing us with 59 different measures. More than 60% of these measures are based on self-reported, subjective ratings and questionnaires (cognitive measurement level) and assess the conscious/decision-making level of emotional processing. This multitude of measures and their overrepresentation in a single category hinders the comparison of results across studies and building a complete multi-faceted picture of food-evoked emotions. We recommend (1) to use widely applied, validated measures only, (2) to refrain from using (highly correlated) measures from the same category but use measures from different categories instead, preferably covering all three emotional processing levels, and (3) to acquire and share simultaneously collected physiological, behavioral, and cognitive datasets to improve the predictive power of food choice and other models.

Non-Contact Heart Rate and Blood Pressure Estimations from Video Analysis and Machine Learning Modelling Applied to Food Sensory Responses: A Case Study for Chocolate

Traditional methods to assess heart rate (HR) and blood pressure (BP) are intrusive and can affect results in sensory analysis of food as participants are aware of the sensors. This paper aims to validate a non-contact method to measure HR using the photoplethysmography (PPG) technique and to develop models to predict the real HR and BP based on raw video analysis (RVA) with an example application in chocolate consumption using machine learning (ML). The RVA used a computer vision algorithm based on luminosity changes on the different RGB color channels using three face-regions (forehead and both cheeks). To validate the proposed method and ML models, a home oscillometric monitor and a finger sensor were used. Results showed high correlations with the G color channel (R² = 0.83). Two ML models were developed using three face-regions: (i) Model 1 to predict HR and BP using the RVA outputs with R = 0.85 and (ii) Model 2 based on time-series prediction with HR, magnitude and luminosity from RVA inputs to HR values every second with R = 0.97. An application for the sensory analysis of chocolate showed significant correlations between changes in HR and BP with chocolate hardness and purchase intention.

O Uso de Biomarcadores em Pesquisas de Marketing

Resumo A aproximação do marketing com a neurociência tem gerado expectativa tanto na academia quanto no mercado, em especial na última década. Apesar do interesse e das promessas de resultados, lacunas de conhecimento neurobiológico são percebidas acerca de formas de coleta e análise de dados. Este estudo apresenta e compara algumas ferramentas neurocientíficas que são fundamentadas em respostas não cognitivas do cérebro humano. A coleta e a análise de biomarcadores, como hormônios e níveis de atividade elétrica muscular, podem ser utilizadas em pesquisas de comportamento humano relacionadas ao consumo ou em outras interações com ações de marketing. A metodologia de aplicação dessas ferramentas, medidas resultantes e possíveis interpretações de comportamento também são discutidas, com ênfase em futura agenda de pesquisa.

Validity of assessing child feeding with virtual reality

Assessment of parents' child feeding behavior is challenging, and there is need for additional methodological approaches. Virtual reality technology allows for the creation of behavioral measures, and its implementation overcomes several limitations of existing methods. This report evaluates the validity and usability of the Virtual Reality (VR) Buffet among a sample of 52 parents of children aged 3-7. Participants served a meal of pasta and apple juice in both a virtual setting and real-world setting (counterbalanced and separated by a distractor task). They then created another meal for their child, this time choosing from the full set of food options in the VR Buffet. Finally, participants completed a food estimation task followed by a questionnaire, which assessed their perceptions of the VR Buffet. Results revealed that the amount of virtual pasta served by parents correlated significantly with the amount of real pasta they served, rs = 0.613, p < .0001, as did served amounts of virtual and real apple juice, rs = 0.822, p < .0001. Furthermore, parents' perception of the calorie content of chosen foods was significantly correlated with observed calorie content (rs = 0.438, p = .002), and parents agreed that they would feed the meal they created to their child (M = 4.43, SD = 0.82 on a 1-5 scale). The data presented here demonstrate that parent behavior in the VR Buffet is highly related to real-world behavior, and that the tool is well-rated by parents. Given the data presented and the potential benefits of the abundant behavioral data the VR Buffet can provide, we conclude that it is a valid and needed addition to the array of tools for assessing feeding behavior.

A comparison of two low-calorie sweeteners and sugar in dark chocolate on sensory attributes and emotional conceptualisations

Reducing sugar consumption is an important aspect in the prevention of and fight against obesity. A broader understanding of consumers' perceptions of low-calorie sweeteners is needed. This study examined two low-calorie sweeteners, tagatose and stevia, in comparison to sugar in dark chocolate. A total of 219 consumers participated in this study and rated overall liking and sensory attributes. Participants also listed their emotional conceptualisations upon consumption and were assessed on emotional eating behaviour and health and taste attitudes. The chocolate with tagatose was perceived as more similar to the chocolate with sugar than with stevia on overall liking, texture, bitterness, duration of aftertaste and intensity of aftertaste. Furthermore, chocolate with sugar and chocolate with tagatose both elicited positive emotional conceptualisations whereas chocolate with stevia elicited negative emotional conceptualisations. In conclusion, dark chocolate with tagatose did not significantly differ from sugar in overall liking, most sensory attributes and emotional conceptualisation.

Predicting consumer liking and preference based on emotional responses and sensory perception: A study with basic taste solutions

Traditional methods of sensory testing focus on capturing information about multisensory perceptions, but do not necessarily measure emotions elicited by these food and beverages. The objective of this study was to develop an optimum model of predicting overall liking (rating) and preference (choice) based on taste intensity and evoked emotions. One hundred and two participants (51 females) were asked to taste water, sucrose, citric acid, salt, and caffeine solutions. Their emotional responses toward each sample were measured by a combination of a self-reported emotion questionnaire (EsSense25), facial expressions, and autonomic nervous system (ANS) responses. In addition, their perceived intensity and overall liking were measured. After a break, participants re-tasted the samples and ranked them according to their preference. The results showed that emotional responses measured using self-reported emotion questionnaire and facial expression analysis along with perceived taste intensity performed best to predict overall liking as well as preference, while ANS measures showed limited contribution. Contrary to some previous research, this study demonstrated that not only negative emotions, but also positive ones could help predict consumer liking and preference. In addition, since there were subtle differences in the prediction models of overall liking and preference, both aspects should be taken into account to understand consumer behavior. In conclusion, combination of evoked emotions along with sensory perception could help better understand consumer acceptance as well as preference toward basic taste solutions.

Asians' Facial Responsiveness to Basic Tastes by Automated Facial Expression Analysis System

Growing evidence shows that consumer choices in real life are mostly driven by unconscious mechanisms rather than conscious. The unconscious process could be measured by behavioral measurements. This study aims to apply automatic facial expression analysis technique for consumers' emotion representation, and explore the relationships between sensory perception and facial responses. Basic taste solutions (sourness, sweetness, bitterness, umami, and saltiness) with 6 levels plus water were used, which could cover most of the tastes found in food and drink. The other contribution of this study is to analyze the characteristics of facial expressions and correlation between facial expressions and perceptive hedonic liking for Asian consumers. Up until now, the facial expression application researches only reported for western consumers, while few related researches investigated the facial responses during food consuming for Asian consumers. Experimental results indicated that facial expressions could identify different stimuli with various concentrations and different hedonic levels. The perceived liking increased at lower concentrations and decreased at higher concentrations, while samples with medium concentrations were perceived as the most pleasant except sweetness and bitterness. High correlations were founded between perceived intensities of bitterness, umami, saltiness, and facial reactions of disgust and fear. Facial expression disgust and anger could characterize emotion "dislike," and happiness could characterize emotion "like," while neutral could represent "neither like nor dislike." The identified facial expressions agree with the perceived sensory emotions elicited by basic taste solutions. The correlation analysis between hedonic levels and facial expression intensities obtained in this study are in accordance with that discussed for western consumers.

Facial expressions and autonomous nervous system responses elicited by tasting different juices

The aim of this study was to get a better understanding of reactions elicited by the taste of foods using the example of different juices. The reactions investigated were the rating behavior of self-reported spontaneous liking, various autonomous nervous system (ANS) responses and implicit as well as explicit facial expressions. Therefore, the following four hypotheses were tested: 1) Different sensory stimuli of juices elicit different ANS responses. 2) Differences in facial expressions elicited by sensory stimuli of juices used in an implicit and explicit measurement approach can be detected by using FaceReader 5. 3) Self-reported liking is correlated with the measured ANS parameters and the elicited facial expressions. 4) The measured ANS parameters, facial expressions and self-reported liking allow identical differentiations between samples. Skin conductance level (SCL), skin temperature (ST), heart rate (HR), pulse volume amplitude (PVA) and the facial expressions of 81 participants were analyzed during and shortly after tasting juice samples (implicit measurement approach). Additionally, participants were asked to show how much they liked the tasted sample with an intentional facial expression (explicit measurement approach). Banana, grapefruit, mixed vegetable, orange and sauerkraut juices were used as sensory stimuli. The juices elicited significant differences in SCL and PVA responses and intensities of several facial expressions. For these parameters a moderate correlation with self-reported liking was found, allowing a differentiation between liked, disliked and neutral rated samples. The results show that self-reported liking cannot simply be explained by the measured ANS and implicit facial expression parameters, instead providing different information. Significant differences in facial expressions between the implicit and explicit approach were observed. In the implicit approach participants showed hardly any positive emotions when tasting samples they liked, whereas in the explicit approach they displayed a high degree of positive emotions. In both cases negative emotions were shown more intensely for disliked samples.

ANS responses and facial expressions differentiate between the taste of commercial breakfast drinks

The high failure rate of new market introductions, despite initial successful testing with traditional sensory and consumer tests, necessitates the development of other tests. This study explored the ability of selected physiological and behavioral measures of the autonomic nervous system (ANS) to distinguish between repeated exposures to foods from a single category (breakfast drinks) and with similar liking ratings. In this within-subject study 19 healthy young adults sipped from five breakfast drinks, each presented five times, while ANS responses (heart rate, skin conductance response and skin temperature), facial expressions, liking, and intensities were recorded. The results showed that liking was associated with increased heart rate and skin temperature, and more neutral facial expressions. Intensity was associated with reduced heart rate and skin temperature, more neutral expressions and more negative expressions of sadness, anger and surprise. Strongest associations with liking were found after 1 second of tasting, whereas strongest associations with intensity were found after 2 seconds of tasting. Future studies should verify the contribution of the additional information to the prediction of market success.

Facial affective reactions to bitter-tasting foods and body mass index in adults

Differences in food consumption among body-weight statuses (e.g., higher fruit intake linked with lower body mass index (BMI) and energy-dense products with higher BMI) has raised the question of why people who are overweight or are at risk of becoming overweight eat differently from thinner people. One explanation, in terms of sensitivity to affective properties of food, suggests that palatability-driven consumption is likely to be an important contributor to food intake, and therefore body weight. Extending this approach to unpalatable tastes, we examined the relationship between aversive reactions to foods and BMI. We hypothesized that people who have a high BMI will show more negative affective reactions to bitter-tasting stimuli, even after controlling for sensory perception differences. Given that hedonic reactions may influence consumption even without conscious feelings of pleasure/displeasure, the facial expressions were included in order to provide more direct access to affective systems than subjective reports. Forty adults (28 females, 12 males) participated voluntarily. Their ages ranged from 18 to 46 years (M=24.2, SD=5.8). On the basis of BMI, participants were classified as low BMI (BMI<20; n=20) and high BMI (BMI>23; n=20). The mean BMI was 19.1 for low BMI (SD=0.7) and 25.2 for high BMI participants (SD=1.8). Each subject tasted 5 mL of a grapefruit juice drink and a bitter chocolate drink. Subjects rated the drinks' hedonic and incentive value, familiarity and bitter intensity immediately after each stimulus presentation. The results indicated that high BMI participants reacted to bitter stimuli showing more profound changes from baseline in neutral and disgust facial expressions compared with low BMI. No differences between groups were detected for the subjective pleasantness and familiarity. The research here is the first to examine how affective facial reactions to bitter food, apart from taste responsiveness, can predict differences in BMI.