Facial expression recognition: A meta-analytic review of theoretical models and neuroimaging evidence

Impairments in facial expression recognition in patients with euthymic bipolar disorders using the facial emotions recognition test

BACKGROUND

Patients with bipolar disorders (BD) have significant impairments in Facial Expression Recognition (FER), an essential social skill for effective social interactions. While the Facial Emotions Recognition Test, 54 photographs by Gaudelus (TREF-54g) has been used in patients with schizophrenia, no study has evaluated FER using this test in BD patients. The TREF-54g meets the International Society for Bipolar Disorders (ISBD) criteria for assessing FER. The objective of this study was to compare FER in patients with euthymic BD versus healthy controls (HC) using the TREF-54g.

METHODS

This study included 60 patients with BD diagnosed according to DSM-5 criteria, 30 with bipolar I disorder (BDI) and 30 with bipolar II disorder (BDII) and 60 HC. All groups were matched for age, gender, and years of education. Overall TREF-54g impairment score was defined as a difference of two SD from the group of HC with mean overall accuracy score expressed as a percentage of correct responses.

RESULTS

Overall TREF-54g impairment scores, as well as sub-scores for anger and contempt recognition, were significantly higher in patients with BD versus those in HC. No significant differences in TREF-54g scores were shown between patients with BDI and those with BDII.

LIMITATIONS

Clinical assessments of this study didn't include neurocognition assessments and classification of contempt emotion as a basic emotion remains controversial.

CONCLUSIONS

We found significant impairments in FER in patients with euthymic BD, particularly for anger and contempt using the TREF-54g a test that meets ISBD recommendation criteria.

"Does facial emotion recognition mediate the relationship between preterm birth and social skills? - A meta-analysis"

BACKGROUND

Preterm birth (PB) is prevalent and associated with structural and functional brain alterations which may affect cognitive and behavioural outcomes, including social development. Facial emotion recognition (FER) is one of the main components of social interaction. PB individuals face distinct FER challenges that may impact social skills. Furthermore, both FER and social skills have shown distinctive developmental trajectories in PB individuals compared to term born controls. This study investigates the association between FER and social skills in PB individuals compared to term-born controls.

OBJECTIVES

To systematically review and meta-analyse relevant literature on the association between FER and social skills and to summarize the reported differences in FER and social skills between PB individuals and term-born controls of similar age.

METHOD

a systematic search of peer-reviewed and English written studies was performed in MEDLINE, Web of Science and CINAHL, with an additional forward and backward citation search. Eligible studies included any observational study that had a term-born control sample of similar age with reported FER and social skills measures and/or correlations between them. Quality assessment and data extraction was carried out. Correlation coefficients and Hedges' g for FER and social skills were calculated as effect size indexes. Random effects model and subgroup analysis considering gestational age and age at assessment was performed. Results were summarized using forest plots. I2 statistics and Cochran's Q were used to test for heterogeneity.

RESULTS

8 studies were included (PB = 410, controls =337). Only 3 studies explored the correlation between FER and social skills. The review found a higher correlation between FER and social skills in the PB group (Z = 0.18, CI = -0.03, 0.39) compared to controls (Z = 0.11, CI = -0.03, 0.25). FER was significantly lower in PB individuals (overall g = -1.48; 95%IC = -2.46, -0-5), particularly in very preterm and adolescent subgroups.

DISCUSSION

FER might play a crucial role in the social development of PB individuals compared to those born at term, but existing research in this domain remains limited.

The Emotion Authenticity Recognition (EAR) test: normative data of an innovative test using dynamic emotional stimuli to evaluate the ability to recognize the authenticity of emotions expressed by faces

Despite research has massively focused on how emotions conveyed by faces are perceived, the perception of emotions' authenticity is a topic that has been surprisingly overlooked. Here, we present the Emotion Authenticity Recognition (EAR) test, a test specifically developed using dynamic stimuli depicting authentic and posed emotions to evaluate the ability of individuals to correctly identify an emotion (emotion recognition index, ER Index) and classify its authenticity (authenticity recognition index (EA Index). The EAR test has been validated on 522 healthy participants and normative values are provided. Correlations with demographic characteristics, empathy and general cognitive status have been obtained revealing that both indices are negatively correlated with age, and positively with education, cognitive status and different facets of empathy. The EAR test offers a new ecological test to assess the ability to detect emotion authenticity that allow to explore the eventual social cognitive deficit even in patients otherwise cognitively intact.

Age- and sex-related differences in social competence and emotion labeling in pre-adolescence

Identification of facial expressions is important to navigate social interactions and associates with developmental outcomes. It is presumed that social competence, behavioral emotion labeling and neural emotional face processing are related, but this has rarely been studied. Here, we investigated these interrelations and their associations with age and sex, in the YOUth cohort (1055 children, 8-11 years old). Using a multistep linear modelling approach, we associated parent-reported social competence, basic emotion labeling skills based on pictures of facial expressions, and neural facial emotion processing during a passive-watching fMRI task with pictures of houses and emotional faces. Results showed better emotion labeling and higher social competence for girls compared to boys. Age was positively associated with emotion labeling skills and specific social competence subscales. These age- and sex-differences were not reflected in brain function. During fMRI, happy faces elicited more activity than neutral or fearful faces. However, we did not find evidence for the hypothesized links between social competence and behavioral emotion labeling, and with neural activity. To conclude, in pre-adolescents, social competence and emotion labeling varied with age and sex, while social competence, emotion labeling and neural processing of emotional faces were not associated with each other.

Facial emotion recognition deficits are associated with hypomimia and related brain correlates in Parkinson's disease

Hypomimia is a frequent manifestation in Parkinson's disease (PD) that can affect interpersonal relationships and quality of life. Recent studies have suggested that hypomimia is not only related to motor dysfunction but also to impairment in emotional processing networks. Therefore, we hypothesized that the severity of hypomimia could be associated with performance on a task aimed at assessing facial emotion recognition. In this study, we explored the association between hypomimia, recognition of facial expressions of basic emotions using the Ekman 60 Faces Test (EF), and brain correlates of both hypomimia and performance on the EF. A total of 94 subjects underwent clinical assessments (neurological and neuropsychological examinations), and 56 of them participated in the neuroimaging study. We found significant correlation between hypomimia, EF Disgust (r = -0.242, p = 0.022) and EF Happiness (r = -0.264, p = 0.012); an independent reduction in Cortical Thickness (Cth) in the postcentral gyrus, insula, middle and superior temporal gyri, supramarginal gyrus, banks of the superior temporal sulcus, bilateral fusiform gyri, entorhinal cortex, parahippocampal gyrus, inferior and superior parietal cortex, and right cuneus and precuneus; and multiple correlations between negative emotions such as EF Disgust or EF Anger and a reduced Cth in fronto-temporo-parietal regions. In conclusion, these results suggest that the association between hypomimia and emotion recognition deficits in individuals with PD might be mediated by shared circuits, supporting the concept that hypomimia is not only the result of the dysfunction of motor circuits, but also of higher cognitive functions.

End-to-End Multimodal Emotion Recognition Based on Facial Expressions and Remote Photoplethysmography Signals

Emotion is a complex physiological phenomenon, and a single modality may be insufficient for accurately determining human emotional states. This paper proposes an end-to-end multimodal emotion recognition method based on facial expressions and non-contact physiological signals. Facial expression features and remote photoplethysmography (rPPG) signals are extracted from facial video data, and a transformer-based cross-modal attention mechanism (TCMA) is used to learn the correlation between the two modalities. The results show that the accuracy of emotion recognition can be slightly improved by combining facial expressions with accurate rPPG signals. The performance is further improved with the use of TCMA, for which the binary classification accuracy of valence and arousal is 91.11% and 90.00%, respectively. Additionally, when experiments are conducted using the whole dataset, an increased accuracy of 7.31% and 4.23% for the binary classification of valence and arousal, and an improved accuracy of 5.36% for the four classifications of valence-arousal are achieved when TCMA is used in modal fusion, compared to using only facial expression modality, which fully demonstrates the effectiveness and robustness of TCMA. This method makes it possible to realize multimodal emotion recognition of facial expressions and contactless physiological signals in reality.

Multisession tDCS combined with intrastimulation training improves emotion recognition in adolescents with autism spectrum disorder

Previous studies indicate that transcranial direct current stimulation (tDCS) is a promising emerging treatment option for autism spectrum disorder (ASD) and its efficacy could be augmented using concurrent training. However, no intrastimulation social cognition training for ASD has been developed so far. The objective of this two-armed, double-blind, randomized, sham-controlled clinical trial is to investigate the effects of tDCS combined with a newly developed intrastimulation social cognition training on adolescents with ASD. Twenty-two male adolescents with ASD were randomly assigned to receive 10 sessions of either anodal or sham tDCS at F3/right supraorbital region together with online intrastimulation training comprising basic and complex emotion recognition tasks. Using baseline magnetic resonance imaging data, individual electric field distributions were simulated, and brain activation patterns of the training tasks were analyzed. Additionally, questionnaires were administered at baseline and following the intervention. Compared to sham tDCS, anodal tDCS significantly improved dynamic emotion recognition over the course of the sessions. This task also showed the highest activations in face processing regions. Moreover, the improvement was associated with electric field density at the medial prefrontal cortex and social awareness in exploratory analyses. Both groups showed high tolerability and acceptability of tDCS, and significant improvement in overall ASD symptoms. Taken together, multisession tDCS improved dynamic emotion recognition in adolescents with ASD using a task that activates brain regions associated with the social brain network. The variability in the electric field might diminish tDCS effects and future studies should investigate individualized approaches.

Facial and Emotion Recognition Deficits in Myasthenia Gravis

Myasthenia gravis (MG) is a neuromuscular disease of autoimmune etiology and chronic evolution. In addition to the muscle weakness and fatigue that characterize MG, in some studies patients show an inferior performance in cognitive tasks and difficulties in recognizing basic emotions from facial expressions. However, it remains unclear if these difficulties are due to anxious-depressive symptoms that these patients present or related to cognitive abilities, such as facial recognition. This study had a descriptive cross-sectional design with a sample of 92 participants, 52 patients with MG and 40 healthy controls. The data collection protocol included measures to assess recognition of facial expressions (BRFT), facial emotional expression (FEEL), and levels of anxiety and depression (HADS). The MG group had worse performance than the control group in recognizing "fear" (p = 0.001; r = 0.344), "happiness" (p = 0.000; r = 0.580), "disgust" (p = 0.000; r = 0.399), "surprise" (p = 0.000; r = 0.602), and "anger" (p = 0.007; r = 0.284). Likewise, the MG group also underperformed in facial recognition (p = 0.001; r = 0.338). These difficulties were not related to their levels of anxiety and depression. Alterations were observed both in the recognition of facial emotions and in facial recognition, without being mediated by emotional variables. These difficulties can influence the interpersonal interaction of patients with MG.

D, B. VS, A. L, M. L. Down-modulation of functional ventral striatum activation for emotional face stimuli in patients with insula damage

Insula damage results in substantial impairments in facial emotion recognition. In particular, left hemispheric damage appears to be associated with poorer recognition of aversively rated facial expressions. Functional imaging can provide information on differences in the processing of these stimuli in patients with insula lesions when compared to healthy matched controls (HCs). We therefore investigated 17 patients with insula lesions in the chronic stage following stroke and 13 HCs using a passive-viewing task with pictures of facial expressions testing the blood oxygenation dependent (BOLD) effect in predefined regions of interest (ROIs). We expected a decrease in functional activation in an area modulating emotional response (left ventral striatum) but not in the facial recognition areas in the left inferior fusiform gyrus. Quantification of BOLD-response in ROIs but also voxel-based statistics confirmed this hypothesis. The voxel-based analysis demonstrated that the decrease in BOLD in the left ventral striatum was driven by left hemispheric damaged patients (n = 10). In our patient group, insula activation was strongly associated with the intensity rating of facial expressions. In conclusion, the combination of performance testing and functional imaging in patients following circumscribed brain damage is a challenging method for understanding emotion processing in the human brain.

Neural correlates of individual differences in multimodal emotion recognition ability

Studies have reported substantial variability in emotion recognition ability (ERA) - an important social skill - but possible neural underpinnings for such individual differences are not well understood. This functional magnetic resonance imaging (fMRI) study investigated neural responses during emotion recognition in young adults (N = 49) who were selected for inclusion based on their performance (high or low) during previous testing of ERA. Participants were asked to judge brief video recordings in a forced-choice emotion recognition task, wherein stimuli were presented in visual, auditory and multimodal (audiovisual) blocks. Emotion recognition rates during brain scanning confirmed that individuals with high (vs low) ERA received higher accuracy for all presentation blocks. fMRI-analyses focused on key regions of interest (ROIs) involved in the processing of multimodal emotion expressions, based on previous meta-analyses. In neural response to emotional stimuli contrasted with neutral stimuli, individuals with high (vs low) ERA showed higher activation in the following ROIs during the multimodal condition: right middle superior temporal gyrus (mSTG), right posterior superior temporal sulcus (PSTS), and right inferior frontal cortex (IFC). Overall, results suggest that individual variability in ERA may be reflected across several stages of decisional processing, including extraction (mSTG), integration (PSTS) and evaluation (IFC) of emotional information.

Early life adversity is associated with greater similarity in neural representations of ambiguous and threatening stimuli

Exposure to early life adversity (ELA) is hypothesized to sensitize threat-responsive neural circuitry. This may lead individuals to overestimate threat in the face of ambiguity, a cognitive-behavioral phenotype linked to poor mental health. The tendency to process ambiguity as threatening may stem from difficulty distinguishing between ambiguous and threatening stimuli. However, it is unknown how exposure to ELA relates to neural representations of ambiguous and threatening stimuli, or how processing of ambiguity following ELA relates to psychosocial functioning. The current fMRI study examined multivariate representations of threatening and ambiguous social cues in 41 emerging adults (aged 18 to 19 years). Using representational similarity analysis, we assessed neural representations of ambiguous and threatening images within affective neural circuitry and tested whether similarity in these representations varied by ELA exposure. Greater exposure to ELA was associated with greater similarity in neural representations of ambiguous and threatening images. Moreover, individual differences in processing ambiguity related to global functioning, an association that varied as a function of ELA. By evidencing reduced neural differentiation between ambiguous and threatening cues in ELA-exposed emerging adults and linking behavioral responses to ambiguity to psychosocial wellbeing, these findings have important implications for future intervention work in at-risk, ELA-exposed populations.

Clinical features in co-occuring obsessive-compulsive disorder and bipolar disorder: A systematic review and meta-analysis

Obsessive-compulsive disorder (OCD) frequently co-occurs with various psychiatric conditions and may impact as many as one-fifth of individuals diagnosed with bipolar disorder (BD). Despite the expanding body of literature on the coexistence of OCD and BD, there is a notable lack of comprehensive data pertaining to the distinct features of obsessive-compulsive symptoms that define this comorbidity. To bridge this knowledge gap, we conducted a systematic search of PubMed/MEDLINE, Scopus, EMBASE, and PsycINFO until August 7th, 2023. We performed random-effects meta-analyses to compare individuals with both OCD and BD to those with OCD in terms of OCD symptomatology as well as the specific categories of obsessions and compulsions. Out of the 10,393 records initially screened, 17 studies were ultimately incorporated into the qualitative assessment, with 15 of them being included in the quantitative analysis. Individuals with OCD and BD experienced fewer lifetime contamination obsessions (OR=0.71; 95 %CI=0.53, 0.95; p = 0.021) and more sexual obsessions (OR=1.77; 95 %CI=1.03, 3.04; p = 0.04) compared to individuals with OCD without BD. No significant difference was observed for other types of obsessions or compulsions or for the severity of OCD symptoms, although BD type may play a role according to meta-regression analyses. The detection of the presence of sexual or contamination obsessions through a detailed interview may be the focus of clinical attention when assessing OCD in the context of comorbid BD. Sub-phenotyping complex clinical presentation of comorbid psychiatric disorders can aid in making more informed decisions when choosing an appropriate treatment approach.

Language access differentially alters functional connectivity during emotion perception across cultures

Introduction

It is often assumed that the ability to recognize the emotions of others is reflexive and automatic, driven only by observable facial muscle configurations. However, research suggests that accumulated emotion concept knowledge shapes the way people perceive the emotional meaning of others' facial muscle movements. Cultural upbringing can shape an individual's concept knowledge, such as expectations about which facial muscle configurations convey anger, disgust, or sadness. Additionally, growing evidence suggests that access to emotion category words, such as "anger," facilitates access to such emotion concept knowledge and in turn facilitates emotion perception.

Methods

To investigate the impact of cultural influence and emotion concept accessibility on emotion perception, participants from two cultural groups (Chinese and White Americans) completed a functional magnetic resonance imaging scanning session to assess functional connectivity between brain regions during emotion perception. Across four blocks, participants were primed with either English emotion category words ("anger," "disgust") or control text (XXXXXX) before viewing images of White American actors posing facial muscle configurations that are stereotypical of anger and disgust in the United States.

Results

We found that when primed with "disgust" versus control text prior to seeing disgusted facial expressions, Chinese participants showed a significant decrease in functional connectivity between a region associated with semantic retrieval (the inferior frontal gyrus) and regions associated with semantic processing, visual perception, and social cognition. Priming the word "anger" did not impact functional connectivity for Chinese participants relative to control text, and priming neither "disgust" nor "anger" impacted functional connectivity for White American participants.

Discussion

These findings provide preliminary evidence that emotion concept accessibility differentially impacts perception based on participants' cultural background.

Differences in facial emotion recognition between bipolar disorder and other clinical populations: A systematic review and meta-analysis

Facial emotion (or expression) recognition (FER) is a domain of affective cognition impaired across various psychiatric conditions, including bipolar disorder (BD). We conducted a systematic review and meta-analysis searching for eligible articles published from inception to April 26, 2023, in PubMed/MEDLINE, Scopus, EMBASE, and PsycINFO to examine whether and to what extent FER would differ between people with BD and those with other mental disorders. Thirty-three studies comparing 1506 BD patients with 1973 clinical controls were included in the present systematic review, and twenty-six of them were analyzed in random-effects meta-analyses exploring the discrepancies in discriminating or identifying emotional stimuli at a general and specific level. Individuals with BD were more accurate in identifying each type of emotion during a FER task compared to individuals diagnosed with schizophrenia (SCZ) (SMD = 0.27; p-value = 0.006), with specific differences in the perception of anger (SMD = 0.46; p-value = 1.19e-06), fear (SMD = 0.38; p-value = 8.2e-04), and sadness (SMD = 0.33; p-value = 0.026). In contrast, BD patients were less accurate than individuals with major depressive disorder (MDD) in identifying each type of emotion (SMD = -0.24; p-value = 0.014), but these differences were more specific for sad emotional stimuli (SMD = -0.31; p-value = 0.009). No significant differences were observed when BD was compared with children and adolescents diagnosed with attention-deficit/hyperactivity disorder. FER emerges as a potential integrative instrument for guiding diagnosis by enabling discrimination between BD and SCZ or MDD. Enhancing the standardization of adopted tasks could further enhance the accuracy of this tool, leveraging FER potential as a therapeutic target.

Alpha-band oscillations and emotion: A review of studies on picture perception

Although alpha-band activity has long been a focus of psychophysiological research, its modulation by emotional value during picture perception has only recently been studied systematically. Here, we review these studies and report that the most consistent alpha oscillatory pattern indexing emotional processing is an enhanced desynchronization (ERD) over posterior sensors when viewing emotional compared with neutral pictures. This enhanced alpha ERD is not specific to unpleasant picture content, as previously proposed for other measures of affective response, but has also been observed for pleasant stimuli. Evidence suggests that this effect is not confined to the alpha band but that it also involves a desynchronization of the lower beta frequencies (8-20 Hz). The emotional modulation of alpha ERD occurs even after massive stimulus repetition and when emotional cues serve as task-irrelevant distractors, consistent with the hypothesis that evaluative processes are mandatory in emotional picture processing. A similar enhanced ERD has been observed for other significant cues (e.g., conditioned aversive stimuli, or in anticipation of a potential threat), suggesting that it reflects cortical excitability associated with the engagement of the motivational systems.

Association Between Frequency of Social Contact and Brain Atrophy in Community-Dwelling Older People Without Dementia: The JPSC-AD Study

BACKGROUND AND OBJECTIVES

Epidemiologic evidence has shown that social isolation, a low frequency of social contact with others, is associated with the risk of dementia and late-life depressive symptoms. Therefore, we hypothesized that low frequency of social contact may be involved in brain atrophy, and depressive symptoms may play some role in this relationship. We aimed to evaluate the association between low frequency of social contact and the volumes of various brain regions and to assess the extent to which depressive symptoms mediate these relationships from a large population-based multisite cohort study.

METHODS

Dementia-free community-dwelling Japanese aged 65 years or older underwent brain MRI scans and a comprehensive health examination. Frequency of contact with noncohabiting relatives and friends was determined by asking a single question with 4 categories: everyday, several times a week, several times a month, and seldom. Total and regional brain volumes, intracranial volume (ICV), and white matter lesion volume were estimated using FreeSurfer software. The associations between frequency of social contact and brain volumes per ICV were examined using analyses of covariance. Mediation analyses were conducted to calculate the proportion of the associations explained by depressive symptoms.

RESULTS

We included 8,896 participants. The multivariable-adjusted mean of the total brain volume in the group with the lowest frequency of social contact was significantly lower compared with that in the group with the highest frequency of social contact (67.3% vs 67.8%), with a significant increasing trend across the groups (p value for trend <0.001). The white matter lesion volume increased significantly with lower frequency of social contact (0.30% in the lowest frequency group vs 0.26% in the highest frequency group, p value for trend <0.001). Lower frequency of social contact was associated with smaller volumes in the temporal lobe, occipital lobe, cingulum, hippocampus, and amygdala (all q values of false discovery rate correction <0.05). The relationships seemed to be partly mediated by depressive symptoms, which accounted for 15%-29% of the observed associations.

DISCUSSION

Lower frequency of social contact was associated with decreased total and cognitive function-related regional brain volumes. In addition, depressive symptoms partially explained the association in community-dwelling older people without dementia in Japan.

Brain structure underlying the empathizing-systemizing difference in children with autism spectrum disorder

BACKGROUND

Behavioral research has shown that children with autism spectrum disorder (ASD) have a higher empathizing-systemizing difference (D score) than normal children. However, there is no research about the neuroanatomical mechanisms of the empathizing-systemizing difference in children with ASD.

METHODS

Participants comprised 41 children with ASD and 39 typically developing (TD) children aged 6‒12 years. Empathizing-systemizing difference was estimated using the D score from the Chinese version of Children's Empathy Quotient and Systemizing Quotient. We quantified brain morphometry, including global and regional brain volumes and surface-based cortical measures (cortical thickness, surface area, and gyrification) via structural magnetic resonance imaging.

RESULTS

We found that the D score was significantly negatively associated with amygdala gray matter volume [β =  -0.16; 95% confidence interval (CI): -0.30,  -0.02; P value = 0.030] in children with ASD. There was a significantly negative association between D score and gyrification in the left lateral occipital cortex (LOC) in children with ASD (B =  -0.10; SE = 0.03; cluster-wise P value = 0.006) and a significantly positive association between D score and gyrification in the right fusiform in TD children (B = 0.10; SE = 0.03; cluster-wise P value = 0.022). Moderation analyses demonstrated significant interactions between D score and diagnosed group in amygdala gray matter volume (β = 0.19; 95% CI 0.04, 0.35; P value = 0.013) and left LOC gyrification (β = 0.11; 95% CI 0.05, 0.17; P value = 0.001) but not in right fusiform gyrification (β = 0.08; 95% CI -0.02, 0.17; P value = 0.105).

CONCLUSIONS

Neuroanatomical variation in amygdala volume and gyrification of LOC could be potential biomarkers for the empathizing-systemizing difference in children with ASD but not in TD children. Large-scale neuroimaging studies are necessary to test the replicability of our findings.

Cognition, prior aggression, and psychopathic traits in relation to impaired multimodal emotion recognition in psychotic spectrum disorders

Background

Psychopathic traits have been associated with impaired emotion recognition in criminal, clinical and community samples. A recent study however, suggested that cognitive impairment reduced the relationship between psychopathy and emotion recognition. We therefore investigated if reasoning ability and psychomotor speed were impacting emotion recognition in individuals with psychotic spectrum disorders (PSD) with and without a history of aggression, as well as in healthy individuals, more than self-rated psychopathy ratings on the Triarchic Psychopathy Measure (TriPM).

Methods

Eighty individuals with PSD (schizophrenia, schizoaffective disorder, delusional disorder, other psychoses, psychotic bipolar disorder) and documented history of aggression (PSD+Agg) were compared with 54 individuals with PSD without prior aggression (PSD-Agg) and with 86 healthy individuals on the Emotion Recognition Assessment in Multiple Modalities (ERAM test). Individuals were psychiatrically stable and in remission from possible substance use disorders. Scaled scores on matrix reasoning, averages of dominant hand psychomotor speed and self-rated TriPM scores were obtained.

Results

Associations existed between low reasoning ability, low psychomotor speed, patient status and prior aggression with total accuracy on the ERAM test. PSD groups performed worse than the healthy group. Whole group correlations between total and subscale scores of TriPM to ERAM were found, but no associations with TriPM scores within each group or in general linear models when accounting for reasoning ability, psychomotor speed, understanding of emotion words and prior aggression.

Conclusion

Self-rated psychopathy was not independently linked to emotion recognition in PSD groups when considering prior aggression, patient status, reasoning ability, psychomotor speed and emotion word understanding.

Revisiting emotion recognition in different types of temporal lobe epilepsy: The influence of facial expression intensity

Temporal lobe epilepsy (TLE) can induce various difficulties in recognizing emotional facial expressions (EFE), particularly for negative valence emotions. However, these difficulties have not been systematically examined according to the localization of the epileptic focus. For this purpose, we used a forced-choice recognition task in which faces expressing fear, sadness, anger, disgust, surprise, or happiness were presented in different intensity levels from moderate to high intensity. The first objective of our study was to evaluate the impact of emotional intensity on the recognition of different categories of EFE in TLE patients compared to control participants. The second objective was to assess the effect of localizationof epileptic focus on the recognition of EFE in patients with medial temporal lobe epilepsy (MTLE) associated or not with hippocampal sclerosis (HS), or lateral temporal lobe epilepsy (LTLE). The results showed that the 272 TLE patients and the 68 control participants were not differently affected by the intensity of EFE. However, we obtained group differences within the clinical population when we took into account the localization of the temporal lobe epileptic focus. As predicted, TLE patients were impaired in recognizing fear and disgust relative to controls. Moreover, the scores of these patients varied according to the localization of the epileptic focus, but not according to the cerebral lateralization of TLE. The facial expression of fear was less well recognized by MTLE patients, with or without HS, and the expression of disgust was less well recognized by LTLE as well as MTLE without HS patients. Moreover, emotional intensity modulated differently the recognition of disgust and surprise of the three patient groups underlying the relevance of using moderate emotional intensity to distinguish the effect of epileptic focus localization. These findings should be taken into account for interpreting the emotional behaviors and deserve to befurther investigated before considering TLE surgical treatment or social cognition interventions in TLE patients.

Interaction between Functional Connectivity and Neural Excitability in Autism: A Novel Framework for Computational Modeling and Application to Biological Data

Functional connectivity (FC) and neural excitability may interact to affect symptoms of autism spectrum disorder (ASD). We tested this hypothesis with neural network simulations, and applied it with functional magnetic resonance imaging (fMRI). A hierarchical recurrent neural network embodying predictive processing theory was subjected to a facial emotion recognition task. Neural network simulations examined the effects of FC and neural excitability on changes in neural representations by developmental learning, and eventually on ASD-like performance. Next, by mapping each neural network condition to subject subgroups on the basis of fMRI parameters, the association between ASD-like performance in the simulation and ASD diagnosis in the corresponding subject subgroup was examined. In the neural network simulation, the more homogeneous the neural excitability of the lower-level network, the more ASD-like the performance (reduced generalization and emotion recognition capability). In addition, in homogeneous networks, the higher the FC, the more ASD-like performance, while in heterogeneous networks, the higher the FC, the less ASD-like performance, demonstrating that FC and neural excitability interact. As an underlying mechanism, neural excitability determines the generalization capability of top-down prediction, and FC determines whether the model's information processing will be top-down prediction-dependent or bottom-up sensory-input dependent. In fMRI datasets, ASD was actually more prevalent in subject subgroups corresponding to the network condition showing ASD-like performance. The current study suggests an interaction between FC and neural excitability, and presents a novel framework for computational modeling and biological application of a developmental learning process underlying cognitive alterations in ASD.

Function without feeling: neural reactivity and intercommunication during flexible motor adjustments evoked by emotional and neutral stimuli

Motor conflicts arise when we need to quickly overwrite prepotent behavior. It has been proposed that affective stimuli modulate the neural processing of motor conflicts. However, previous studies have come to inconsistent conclusions regarding the neural impact of affective information on conflict processing. We employed functional magnetic resonance imaging during a Go/Change-Go task, where motor conflicts were either evoked by neutral or emotionally negative stimuli. Dynamic causal modeling was used to investigate how motor conflicts modulate the intercommunication between the anterior cingulate cortex (ACC) and the anterior insula (AI) as 2 central regions for cognitive control. Conflicts compared to standard actions were associated with increased BOLD activation in several brain areas, including the dorsal ACC and anterior insula. There were no differences in neural activity between emotional and non-emotional conflict stimuli. Conflicts compared to standard actions lowered neural self-inhibition of the ACC and AI and led to increased effective connectivity from the ACC to AI contralateral to the acting hand. Thus, our study indicates that neural conflict processing is primarily driven by the functional relevance of action-related stimuli, not their inherent affective meaning. Furthermore, it sheds light on the role of interconnectivity between ACC and AI for the implementation of flexible behavioral change.

Decoding six basic emotions from brain functional connectivity patterns

Although distinctive neural and physiological states are suggested to underlie the six basic emotions, basic emotions are often indistinguishable from functional magnetic resonance imaging (fMRI) voxelwise activation (VA) patterns. Here, we hypothesize that functional connectivity (FC) patterns across brain regions may contain emotion-representation information beyond VA patterns. We collected whole-brain fMRI data while human participants viewed pictures of faces expressing one of the six basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) or showing neutral expressions. We obtained FC patterns for each emotion across brain regions over the whole brain and applied multivariate pattern decoding to decode emotions in the FC pattern representation space. Our results showed that the whole-brain FC patterns successfully classified not only the six basic emotions from neutral expressions but also each basic emotion from other emotions. An emotion-representation network for each basic emotion that spanned beyond the classical brain regions for emotion processing was identified. Finally, we demonstrated that within the same brain regions, FC-based decoding consistently performed better than VA-based decoding. Taken together, our findings revealed that FC patterns contained emotional information and advocated for paying further attention to the contribution of FCs to emotion processing.

Meta-analytic evidence for the cognitive control model of loneliness in emotion processing

Loneliness is strongly related to affective dysregulation. However, the neuropsychological mechanisms underpinning the loneliness-affective processing relationships remain unclear. Here, we first utilised the coordinate-based activation likelihood estimation method to confirm functional clusters related to loneliness, including the striatum, superior and medial frontal gyrus, insula, and cuneus. Meta-analytic connectivity modelling was then performed to characterise the functional connectivity of these clusters across studies using emotion tasks. Our results revealed that these clusters co-activated with the cognitive control networks. From the literature, we understand that loneliness and its neural correlates are highly related to regulating the attention biases to social rewards and social cues. Therefore, our findings provide a proof-of-concept that loneliness up-regulates the cognitive control networks to process socio-affective information. Prolonged up-regulation thus exhausts cognitive resources and hence, affective dysregulation. This study offers insight into the intricate role of cognitive and affective regulation in loneliness and social perception and provides meta-analytic evidence of the cognitive control model of loneliness and loneliness-related affective dysregulation, bringing significant clinical implications.

Optimal Compact Network for Micro-Expression Analysis System

Micro-expression analysis is the study of subtle and fleeting facial expressions that convey genuine human emotions. Since such expressions cannot be controlled, many believe that it is an excellent way to reveal a human’s inner thoughts. Analyzing micro-expressions manually is a very time-consuming and complicated task, hence many researchers have incorporated deep learning techniques to produce a more efficient analysis system. However, the insufficient amount of micro-expression data has limited the network’s ability to be fully optimized, as overfitting is likely to occur if a deeper network is utilized. In this paper, a complete deep learning-based micro-expression analysis system is introduced that covers the two main components of a general automated system: spotting and recognition, with also an additional element of synthetic data augmentation. For the spotting part, an optimized continuous labeling scheme is introduced to spot the apex frame in a video. Once the apex frames have been recognized, they are passed to the generative adversarial network to produce an additional set of augmented apex frames. Meanwhile, for the recognition part, a novel convolutional neural network, coined as Optimal Compact Network (OC-Net), is introduced for the purpose of emotion recognition. The proposed system achieved the best F1-score of 0.69 in categorizing the emotions with the highest accuracy of 79.14%. In addition, the generated synthetic data used in the training phase also contributed to performance improvement of at least 0.61% for all tested networks. Therefore, the proposed optimized and compact deep learning system is suitable for mobile-based micro-expression analysis to detect the genuine human emotions.

Multimodal emotion perception in young and elderly patients with multiple sclerosis

BACKGROUND

Studies suggest that emotion recognition and empathy are impaired in patients with MS (pwMS). Nonetheless, most studies are restricted to young samples, to facial emotion recognition and to self-report assessments of empathy. The aims of this study are to determine the impact of MS and age on multimodal emotion recognition (facial emotions and vocal emotional bursts) and on socioemotional sensitivity (as reported by the participants and their informants). We also aim to investigate the associations between emotion recognition, socioemotional sensitivity, and cognitive measures.

METHODS

We recruited 13 young healthy controls (HC), 14 young pwMS, 14 elderly HC and 15 elderly pwMS. They underwent a short neuropsychological battery, an experimental emotion recognition task including facial emotions and vocal emotional bursts. Both participants and their study informants completed the Revised-Self Monitoring Scale (RSMS) to assess the participant's socioemotional sensitivity.

RESULTS

There was a significant effect of age and group on recognition of both facial emotions and emotional vocal bursts, HC performing significantly better than pwMS, and young participants performing better than elderly participants (no interaction effect). The same effects were observed on self-reported socioemotional sensitivity. However, lower socioemotional sensitivity in pwMS was not reported by the informants. Finally, multimodal emotion recognition did not correlate with socioemotional sensitivity, but it correlated with global cognitive severity.

CONCLUSION

PwMS present with multimodal emotion perception deficits. Our results extend previous findings of decreased emotion perception and empathy to a group of elderly pwMS, in which advancing age does not accentuate these deficits. However, the decreased socioemotional sensitivity reported by pwMS does not appear to be observed by their relatives, nor to correlate with their emotion perception impairments. Future studies should investigate the real-life impacts of emotion perception deficits in pwMS.