Effects of facial biofeedback on hypomimia, emotion recognition, and affect in Parkinson's disease

OBJECTIVES

Facial expressions are a core component of emotions and nonverbal social communication. Therefore, hypomimia as secondary symptom of Parkinson's disease (PD) has adverse effects like social impairment, stigmatization, under-diagnosis and under-treatment of depression, and a generally lower quality of life. Beside unspecific dopaminergic treatment, specific treatment options for hypomimia in PD are rarely investigated. This quasi-randomized controlled trial evaluated the short-term effects of facial electromyogram (EMG) based biofeedback to enhance facial expression and emotion recognition as nonverbal social communication skills in PD patients. Furthermore effects on affect are examined.

METHOD

A sample of 34 in-patients with PD were allocated either to facial EMG-biofeedback as experimental group or non-facial exercises as control group. Facial expression during posing of emotions (measured via EMG), facial emotion recognition, and positive and negative affect were assessed before and after treatment. Stronger improvements were expected in the EMG-biofeedback in comparison to the control group.

RESULTS

The facial EMG-biofeedback group showed significantly greater improvements in overall facial expression, and especially for happiness and disgust. Also, overall facial emotion recognition abilities improved significantly stronger in the experimental group. Positive affect was significantly increased in both groups with no significant differences between them, while negative affect did not change within both groups.

CONCLUSIONS

The study provides promising evidence for facial EMG-biofeedback as a tool to improve facial expression and emotion recognition in PD. Embodiment theories are discussed as working mechanism.

Autonomic and facial electromyographic responses to watching eyes

We measured participants' psychophysiological responses and gaze behavior while viewing a stimulus person's direct and averted gaze in three different conditions manipulating the participants' experience of being watched. The results showed that skin conductance responses and heart rate deceleration responses were greater to direct than averted gaze only in the condition in which the participants had the experience of being watched by the other individual. In contrast, gaze direction had no effects on these responses when the participants were manipulated to believe that the other individual could not watch them or when the stimulus person was presented in a pre-recorded video. Importantly, the eye tracking measures showed no differences in participants' looking behavior between these stimulus presentation conditions. The results of facial electromyography responses suggested that direct gaze elicited greater zygomatic and periocular responses than averted gaze did, independent of the presentation condition. It was concluded that the affective arousal and attention-orienting indexing autonomic responses to eye contact are driven by the experience of being watched. In contrast, the facial responses seem to reflect automatized affiliative responses which can be elicited even in conditions in which seeing another's direct gaze does not signal that the self is being watched.

Identifying the Optimal Location of Facial EMG for Emotion Detection Using Logistic Regression

In this study, we analyzed the utility of electromyogram (EMG) signals recorded from the zygomaticus major (zEMG), the trapezius (tEMG), and the corrugator supercilii (cEMG) for emotion detection. We computed eleven-time domain features from the EMG signals to classify the emotions such as amusing, boring, relaxing, and scary. The features were fed to the logistic regression, support vector machine, and multilayer perceptron classifiers, and model performance was evaluated. We achieved an average 10-fold cross-validation classification accuracy of 67.29%. 67.92% and 64.58% by LR using the features extracted from the EMG signals recorded from the zEMG, tEMG, and cEMG, respectively. The classification accuracy improved to 70.6% while combining features from the zEMG and cEMG for the LR model. However, the performance dropped while including the features of EMG from all three locations. Our study shows the importance of utilizing the zEMG and cEMG combination for emotion recognition.

Synchrony with distress in affective empathy and compassion

Sensitivity to suffering of others is a core factor in social cohesion and evolutionary success. The emergence of such sensitivity may occur via two neuro-functional mechanisms. One is sharing the pain and distress of others, which relies on affective empathy. The other involves a caring concern for others' wellbeing, termed compassion. Both affective empathy and compassion are triggered by cues of pain and distress, exhibited by suffering targets. Yet, the mechanisms underlying distress processing in empathy and compassion are not clear. In the current research, we investigated synchrony with a target's distress, as a putative mechanism for continuous processing of distress cues. Participants viewed a video of a target in distress when given two different instructions: they were asked to continuously rate their distress in the affective empathy condition, or their feelings of care in the compassion condition. We used these dynamic ratings as well as participants' autonomic and facial responses to assess multi-channel synchrony with the target's self-rated distress fluctuations. Dynamic ratings and facial corrugator responses were significantly positively synchronized with the target's distress. For the corrugator responses, synchrony with the target was more pronounced than synchrony with participants' own ratings. Autonomic responses exhibited negative synchrony with the target's distress. Synchrony was higher in the affective empathy than in the compassion condition, across channels. These results point to the key role of subjective and physiological synchrony with the target's distress in empathic sharing of negative experiences. They also highlight the attenuation of embodied resonance with distress in compassionate experiences.

Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback

Patients who have lost limb control ability, such as upper limb amputation and high paraplegia, are usually unable to take care of themselves. Establishing a natural, stable, and comfortable human-computer interface (HCI) for controlling rehabilitation assistance robots and other controllable equipments will solve a lot of their troubles. In this study, a complete limbs-free face-computer interface (FCI) framework based on facial electromyography (fEMG) including offline analysis and online control of mechanical equipments was proposed. Six facial movements related to eyebrows, eyes, and mouth were used in this FCI. In the offline stage, 12 models, eight types of features, and three different feature combination methods for model inputing were studied and compared in detail. In the online stage, four well-designed sessions were introduced to control a robotic arm to complete drinking water task in three ways (by touch screen, by fEMG with and without audio feedback) for verification and performance comparison of proposed FCI framework. Three features and one model with an average offline recognition accuracy of 95.3%, a maximum of 98.8%, and a minimum of 91.4% were selected for use in online scenarios. In contrast, the way with audio feedback performed better than that without audio feedback. All subjects completed the drinking task in a few minutes with FCI. The average and smallest time difference between touch screen and fEMG under audio feedback were only 1.24 and 0.37 min, respectively.

Facing the enemy: Spontaneous facial reactions towards suffering opponents

The suffering of an opponent is an important social affective cue that modulates how aggressive interactions progress. To investigate the affective consequences of opponent suffering on a revenge seeking individual, two experiments (total N = 82) recorded facial muscle activity while participants observed the reaction of a provoking opponent to a (retaliatory) sound punishment in a laboratory aggression task. Opponents reacted via prerecorded videos either with facial displays of pain, sadness, or neutrality. Results indicate that participants enjoyed seeing the provocateur suffer: indexed by a coordinated muscle response featuring an increase in zygomaticus major (and orbicularis oculi muscle) activation accompanied by a decrease in corrugator supercilii activation. This positive facial reaction was only shown while a provoking opponent expressed pain. Expressions of sadness, and administration of sound blasts to nonprovoking opponents, did not modulate facial activity. Overall, the results suggest that revenge-seeking individuals enjoy observing the offender suffer, which could represent schadenfreude or satisfaction of having succeeded in the retaliation goal.

Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces

Decoding someone's facial expressions provides insights into his or her emotional experience. Recently, Automatic Facial Coding (AFC) software has been developed to provide measurements of emotional facial expressions. Previous studies provided first evidence for the sensitivity of such systems to detect facial responses in study participants. In the present experiment, we set out to generalise these results to affective responses as they can occur in variable social interactions. Thus, we presented facial expressions (happy, neutral, angry) and instructed participants (N = 64) to either actively mimic, to look at them passively (n = 21), or to inhibit their own facial reaction (n = 22). A video stream for AFC and an electromyogram (EMG) of the zygomaticus and corrugator muscles were registered continuously. In the mimicking condition, both AFC and EMG differentiated well between facial expressions in response to the different emotional pictures. In the passive viewing and in the inhibition condition AFC did not detect changes in facial expressions whereas EMG was still highly sensitive. Although only EMG is sensitive when participants intend to conceal their facial reactions, these data extend previous findings that Automatic Facial Coding is a promising tool for the detection of intense facial reaction.

Climbing up or falling down: Narcissism predicts physiological sensitivity to social status in children and their parents

Children's narcissism may be rooted in sensitivity to social status (i.e., prominence, respect, and influence in a social group), and this sensitivity might be shared with parents. Testing this idea, a randomized experiment examined how children with high narcissism levels and their parents respond to gains and losses of social status. On a simulated social media platform, children (N = 123, ages 8–13) competed with fictitious peers for status and were randomly assigned to gain or lose status. Unbeknownst to children, parents viewed the course of the task. Children's and parents' affective reactions during the task were measured with facial electromyography, which detects spontaneous facial muscle activity linked to positive affect (i.e., zygomaticus major activity, involved in smiling) and negative affect (i.e., corrugator supercilii activity, involved in frowning). Children with higher narcissism levels showed steeper increases in negative affect during status loss and steeper increases in both positive and negative affect during status gain. Their parents mirrored the steeper increase in positive affect during their child's status gain, but they did not mirror the increase in negative affect. These results suggest that children with high narcissism levels and their parents show intensified affective‐motivational responses to children's status‐relevant experiences. These responses may be transmitted from one generation to the other (e.g., genetically or through parent–child socialization).

Read My Face: Automatic Facial Coding Versus Psychophysiological Indicators of Emotional Valence and Arousal

Facial expressions provide insight into a person's emotional experience. To automatically decode these expressions has been made possible by tremendous progress in the field of computer vision. Researchers are now able to decode emotional facial expressions with impressive accuracy in standardized images of prototypical basic emotions. We tested the sensitivity of a well-established automatic facial coding software program to detect spontaneous emotional reactions in individuals responding to emotional pictures. We compared automatically generated scores for valence and arousal of the Facereader (FR; Noldus Information Technology) with the current psychophysiological gold standard of measuring emotional valence (Facial Electromyography, EMG) and arousal (Skin Conductance, SC). We recorded physiological and behavioral measurements of 43 healthy participants while they looked at pleasant, unpleasant, or neutral scenes. When viewing pleasant pictures, FR Valence and EMG were both comparably sensitive. However, for unpleasant pictures, FR Valence showed an expected negative shift, but the signal differentiated not well between responses to neutral and unpleasant stimuli, that were distinguishable with EMG. Furthermore, FR Arousal values had a stronger correlation with self-reported valence than with arousal while SC was sensitive and specifically associated with self-reported arousal. This is the first study to systematically compare FR measurement of spontaneous emotional reactions to standardized emotional images with established psychophysiological measurement tools. This novel technology has yet to make strides to surpass the sensitivity of established psychophysiological measures. However, it provides a promising new measurement technique for non-contact assessment of emotional responses.

The impact of automatic evaluation on mood: an awareness-dependent effect

How do affective feelings arise? Most theories consider that affective feelings result from the appraisals of an event, these appraisals being the consequences of automatic evaluations processes that can occur pre-consciously. However, studies testing this hypothesis have provided divergent results. We hypothesised that the discrepancy could be due to the visibility of the affective stimuli. We conducted two studies in which we manipulated this factor. Moreover, to exclude a possible explanation of these effects in terms of semantic priming or of experimental demand, we measured participants' facial expressions with EMG in addition to self-report measures (BMIS). The two studies showed that repeated brief exposure to visible positive stimuli (words, Experiment 1 - pictures, Experiment 2) led to more positive feelings than exposure to negative stimuli. In contrast, when the stimuli were masked, the reversed pattern of results was observed. Results on facial EMG indicate similar effects. Consistent with the automatic evaluation hypothesis, the results suggest that assimilative effects of affective stimuli on affective feelings can be observed after a repeated brief presentation of primes, provided that the stimuli are visible. The contrast effects observed when stimuli were masked appear reliable but remain in need for further theoretical explanation.

Apology and Restitution: The Psychophysiology of Forgiveness After Accountable Relational Repair Responses

Apology and restitution each represents wrongdoers' accountable repair responses that have promoted victims' self-reported empathy and forgiveness in crime scenario research. The current study measured emotional and stress-related dependent variables including physiological measures, to illuminate the links between predictors of forgiveness and health-relevant side effects. Specifically, we tested the independent and interactive effects of apology and restitution on forgiveness, emotion self-reports, and facial responses, as well as cardiac measures associated with stress in 32 males and 29 females. Apology and restitution each independently increased empathy, forgiveness, gratitude, and positive emotions, while reducing unforgiveness, negative emotion, and muscle activity above the brow (corrugator supercilii, CS). The presence of a thorough apology-regardless of whether restitution was present-also calmed heart rate, reduced rate pressure products indicative of cardiac stress, and decreased muscle activity under the eye (orbicularis oculi, OO). Interactions pointed to the more potent effects of restitution compared to apology for reducing unforgiveness and anger, while elevating positivity and gratitude. The findings point to distinctive impacts of apology and restitution as factors that foster forgiveness, along with emotional and embodied changes relevant to health.

Emotional misattribution: Facial muscle responses partially mediate behavioral responses in the emotion misattribution procedure

There is ongoing debate regarding the degree to which, and the conditions under which, physiological, affect-related (i.e., embodied) processes contribute to emotion information processing. Whereas most studies focus on clearly visible and intentional processing conditions, the present study targeted this issue by studying the implicit processing of emotional (angry, fearful, joyful, neutral) faces in a masked emotion misattribution procedure. That is, participants had to categorize neutral-looking faces with regard to the allegedly felt emotion, which were preceded by a very briefly presented emotional expression. In addition to behavioral measures, facial muscle responses were obtained as an index of physiological, affect-related processes. Linear mixed-model mediation analyses confirmed that facial muscle responses partially mediated the behavioral responses to the masked primes in the misattribution task.

Sex Differences in Affective Facial Reactions Are Present in Childhood

Adults exposed to affective facial displays produce specific rapid facial reactions (RFRs) which are of lower intensity in males compared to females. We investigated such sex difference in a population of 60 primary school children (30 F; 30 M), aged 7–10 years. We recorded the surface electromyographic (EMG) signal from the corrugator supercilii and the zygomatici muscles, while children watched affective facial displays. Results showed the expected smiling RFR to smiling faces and the expected frowning RFR to sad faces. A systematic difference between male and female participants was observed, with boys showing less ample EMG responses than age-matched girls. We demonstrate that sex differences in the somatic component of affective motor patterns are present also in childhood.

Enhanced Positive Emotional Reactivity Undermines Empathy in Behavioral Variant Frontotemporal Dementia

Behavioral variant frontotemporal dementia (bvFTD) is a neurodegenerative disease characterized by profound changes in emotions and empathy. Although most patients with bvFTD become less sensitive to negative emotional cues, some patients become more sensitive to positive emotional stimuli. We investigated whether dysregulated positive emotions in bvFTD undermine empathy by making it difficult for patients to share (emotional empathy), recognize (cognitive empathy), and respond (real-world empathy) to emotions in others. Fifty-one participants (26 patients with bvFTD and 25 healthy controls) viewed photographs of neutral, positive, negative, and self-conscious emotional faces and then identified the emotions displayed in the photographs. We used facial electromyography to measure automatic, sub-visible activity in two facial muscles during the task: Zygomaticus major (ZM), which is active during positive emotional reactions (i.e., smiling), and Corrugator supercilii (CS), which is active during negative emotional reactions (i.e., frowning). Participants rated their baseline positive and negative emotional experience before the task, and informants rated participants' real-world empathic behavior on the Interpersonal Reactivity Index. The majority of participants also underwent structural magnetic resonance imaging. A mixed effects model found a significant diagnosis X trial interaction: patients with bvFTD showed greater ZM reactivity to neutral, negative (disgust and surprise), self-conscious (proud), and positive (happy) faces than healthy controls. There was no main effect of diagnosis or diagnosis X trial interaction on CS reactivity. Compared to healthy controls, patients with bvFTD had impaired emotion recognition. Multiple regression analyses revealed that greater ZM reactivity predicted worse negative emotion recognition and worse real-world empathy. At baseline, positive emotional experience was higher in bvFTD than healthy controls and also predicted worse negative emotion recognition. Voxel-based morphometry analyses found that smaller volume in the thalamus, midcingulate cortex, posterior insula, anterior temporal pole, amygdala, precentral gyrus, and inferior frontal gyrus—structures that support emotion generation, interoception, and emotion regulation—was associated with greater ZM reactivity in bvFTD. These findings suggest that dysregulated positive emotional reactivity may relate to reduced empathy in bvFTD by making patients less likely to tune their reactions to the social context and to share, recognize, and respond to others' feelings and needs.

Emotion regulation in heavy smokers: experiential, expressive and physiological consequences of cognitive reappraisal

Emotion regulation dysfunctions are assumed to contribute to the development of tobacco addiction and relapses among smokers attempting to quit. To further examine this hypothesis, the present study compared heavy smokers with non-smokers (NS) in a reappraisal task. Specifically, we investigated whether non-deprived smokers (NDS) and deprived smokers (DS) differ from non-smokers in cognitive emotion regulation and whether there is an association between the outcome of emotion regulation and the cigarette craving. Sixty-five participants (23 non-smokers, 22 NDS, and 20 DS) were instructed to down-regulate emotions by reappraising negative or positive pictorial scenarios. Self-ratings of valence, arousal, and cigarette craving as well as facial electromyography and electroencephalograph activities were measured. Ratings, facial electromyography, and electroencephalograph data indicated that both NDS and DS performed comparably to nonsmokers in regulating emotional responses via reappraisal, irrespective of the valence of pictorial stimuli. Interestingly, changes in cigarette craving were positively associated with regulation of emotional arousal irrespective of emotional valence. These results suggest that heavy smokers are capable to regulate emotion via deliberate reappraisal and smokers’ cigarette craving is associated with emotional arousal rather than emotional valence. This study provides preliminary support for the therapeutic use of reappraisal to replace maladaptive emotion-regulation strategies in nicotine addicts.

Gender differences in experiential and facial reactivity to approval and disapproval during emotional social interactions

Negative social evaluations represent social threats and elicit negative emotions such as anger or fear. Positive social evaluations, by contrast, may increase self-esteem and generate positive emotions such as happiness and pride. Gender differences are likely to shape both the perception and expression of positive and negative social evaluations. Yet, current knowledge is limited by a reliance on studies that used static images of individual expressers with limited external validity. Furthermore, only few studies considered gender differences on both the expresser and perceiver side. The present study approached these limitations by utilizing a naturalistic stimulus set displaying nine males and nine females (expressers) delivering social evaluative sentences to 32 female and 26 male participants (perceivers). Perceivers watched 30 positive, 30 negative, and 30 neutral messages while facial electromyography (EMG) was continuously recorded and subjective ratings were obtained. Results indicated that men expressing positive evaluations elicited stronger EMG responses in both perceiver genders. Arousal was rated higher when positive evaluations were expressed by the opposite gender. Thus, gender differences need to be more explicitly considered in research of social cognition and affective science using naturalistic social stimuli.

Frowning and jaw clenching muscle activity reflects the perception of effort during incremental workload cycling

The present study aimed to investigate whether facial electromyography (EMG) recordings reflect the perception of effort and primary active lower limb muscle activity during incremental workload cycling. The effects of exercise intensity on EMG activity of the corrugator supercilii (CS), masseter and vastus lateralis (VL) muscles, heart rate (HR) and the rating of perceived exertion (RPE) were investigated, and the correlations among these parameters were determined. Eighteen males and 15 females performed continuous incremental workload cycling exercise until exhaustion. CS, masseter and VL muscle activities were continuously recorded using EMG during exercise. HR was also continuously monitored during the test. During the final 30 s of each stage of cycle ergometer exercise, participants were asked to report their feeling of exertion on the adult OMNI-Cycle RPE. HR and EMG activity of the facial muscles and the primary active lower limb muscle were strongly correlated with RPE; they increased with power output. Furthermore, facial muscle activity increased significantly during high-intensity exercise. Masseter muscle activity was strongly and positively correlated with HR, RPE and VL activity. The present investigation supports the view that facial EMG activity reflects the perception of effort. The jaw clenching facial expression can be considered an important factor for improving the reporting of perceived effort during high-intensity exercise in males and females. Key pointsFrowning and jaw clenching muscle activity reflects the perception of effort during incremental workload cycling.EMG activity of the masseter muscle was strongly and positively correlated with RPE, HR and lower limb EMG activity during incremental workload cycling.The jaw clenching facial expression can be considered an important factor for estimating the intensity of effort.

Contextual modulation of physiological and psychological responses triggered by emotional stimuli

A series of emotional events successively occur in temporal context. The present study investigated how physiological and psychological responses are modulated by emotional context. Skin conductance response (SCR), heart rate, corrugator activity, zygomatic activity, and subjective feelings during emotional picture viewing were measured. To create an emotional context, a unpleasant or pleasant picture was preceded by three types of pictures, i.e., unpleasant, pleasant, and neutral pictures, resulting in six pairings. The results showed that viewing an unpleasant picture attenuated pleasant feelings induced by the following pleasant picture. On the other hand, preceding pleasant pictures decreased SCR to the following pictures. The effects of contextual modulation on emotional responses might be due to the informative function of pre-existing feelings; unpleasant feelings signal a threatening environment, whereas pleasant feelings signal a benign environment. With respect to facial muscle activities, viewing a pleasant picture decreased corrugator activity in response to the preceding picture. These findings suggest several types of contextual modulation effects on psychological, autonomic, and somatic responses to emotional stimuli.