Affective Computing and the Impact of Gender and Age

Culture and gender modulate dlPFC integration in the emotional brain: evidence from dynamic causal modeling

Past research has recognized culture and gender variation in the experience of emotion, yet this has not been examined on a level of effective connectivity. To determine culture and gender differences in effective connectivity during emotional experiences, we applied dynamic causal modeling (DCM) to electroencephalography (EEG) measures of brain activity obtained from Chinese and American participants while they watched emotion-evoking images. Relative to US participants, Chinese participants favored a model bearing a more integrated dorsolateral prefrontal cortex (dlPFC) during fear v. neutral experiences. Meanwhile, relative to males, females favored a model bearing a less integrated dlPFC during fear v. neutral experiences. A culture-gender interaction for winning models was also observed; only US participants showed an effect of gender, with US females favoring a model bearing a less integrated dlPFC compared to the other groups. These findings suggest that emotion and its neural correlates depend in part on the cultural background and gender of an individual. To our knowledge, this is also the first study to apply both DCM and EEG measures in examining culture-gender interaction and emotion.

Bosses without a heart: socio-demographic and cross-cultural determinants of attitude toward Emotional AI in the workplace

Biometric technologies are becoming more pervasive in the workplace, augmenting managerial processes such as hiring, monitoring and terminating employees. Until recently, these devices consisted mainly of GPS tools that track location, software that scrutinizes browser activity and keyboard strokes, and heat/motion sensors that monitor workstation presence. Today, however, a new generation of biometric devices has emerged that can sense, read, monitor and evaluate the affective state of a worker. More popularly known by its commercial moniker, Emotional AI, the technology stems from advancements in affective computing. But whereas previous generations of biometric monitoring targeted the exterior physical body of the worker, concurrent with the writings of Foucault and Hardt, we argue that emotion-recognition tools signal a far more invasive disciplinary gaze that exposes and makes vulnerable the inner regions of the worker-self. Our paper explores attitudes towards empathic surveillance by analyzing a survey of 1015 responses of future job-seekers from 48 countries with Bayesian statistics. Our findings reveal affect tools, left unregulated in the workplace, may lead to heightened stress and anxiety among disadvantaged ethnicities, gender and income class. We also discuss a stark cross-cultural discrepancy whereby East Asians, compared to Western subjects, are more likely to profess a trusting attitude toward EAI-enabled automated management. While this emerging technology is driven by neoliberal incentives to optimize the worksite and increase productivity, ultimately, empathic surveillance may create more problems in terms of algorithmic bias, opaque decisionism, and the erosion of employment relations. Thus, this paper nuances and extends emerging literature on emotion-sensing technologies in the workplace, particularly through its highly original cross-cultural study.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00146-021-01290-1.

A Review of AI Cloud and Edge Sensors, Methods, and Applications for the Recognition of Emotional, Affective and Physiological States

Affective, emotional, and physiological states (AFFECT) detection and recognition by capturing human signals is a fast-growing area, which has been applied across numerous domains. The research aim is to review publications on how techniques that use brain and biometric sensors can be used for AFFECT recognition, consolidate the findings, provide a rationale for the current methods, compare the effectiveness of existing methods, and quantify how likely they are to address the issues/challenges in the field. In efforts to achieve the key goals of Society 5.0, Industry 5.0, and human-centered design better, the recognition of emotional, affective, and physiological states is progressively becoming an important matter and offers tremendous growth of knowledge and progress in these and other related fields. In this research, a review of AFFECT recognition brain and biometric sensors, methods, and applications was performed, based on Plutchik's wheel of emotions. Due to the immense variety of existing sensors and sensing systems, this study aimed to provide an analysis of the available sensors that can be used to define human AFFECT, and to classify them based on the type of sensing area and their efficiency in real implementations. Based on statistical and multiple criteria analysis across 169 nations, our outcomes introduce a connection between a nation's success, its number of Web of Science articles published, and its frequency of citation on AFFECT recognition. The principal conclusions present how this research contributes to the big picture in the field under analysis and explore forthcoming study trends.

Trends in Heart-Rate Variability Signal Analysis

Heart rate variability (HRV) is the rate of variability between each heartbeat with respect to time. It is used to analyse the Autonomic Nervous System (ANS), a control system used to modulate the body's unconscious action such as cardiac function, respiration, digestion, blood pressure, urination, and dilation/constriction of the pupil. This review article presents a summary and analysis of various research works that analyzed HRV associated with morbidity, pain, drowsiness, stress and exercise through signal processing and machine learning methods. The points of emphasis with regards to HRV research as well as the gaps associated with processes which can be improved to enhance the quality of the research have been discussed meticulously. Restricting the physiological signals to Electrocardiogram (ECG), Electrodermal activity (EDA), photoplethysmography (PPG), and respiration (RESP) analysis resulted in 25 articles which examined the cause and effect of increased/reduced HRV. Reduced HRV was generally associated with increased morbidity and stress. High HRV normally indicated good health, and in some instances, it could signify clinical events of interest such as drowsiness. Effective analysis of HRV during ambulatory and motion situations such as exercise, video gaming, and driving could have a significant impact toward improving social well-being. Detection of HRV in motion is far from perfect, situations involving exercise or driving reported accuracy as high as 85% and as low as 59%. HRV detection in motion can be improved further by harnessing the advancements in machine learning techniques.

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

New types of artificial intelligence products are gradually transferring to voice interaction modes with the demand for intelligent products expanding from communication to recognizing users' emotions and instantaneous feedback. At present, affective acoustic models are constructed through deep learning and abstracted into a mathematical model, making computers learn from data and equipping them with prediction abilities. Although this method can result in accurate predictions, it has a limitation in that it lacks explanatory capability; there is an urgent need for an empirical study of the connection between acoustic features and psychology as the theoretical basis for the adjustment of model parameters. Accordingly, this study focuses on exploring the differences between seven major “acoustic features” and their physical characteristics during voice interaction with the recognition and expression of “gender” and “emotional states of the pleasure-arousal-dominance (PAD) model.” In this study, 31 females and 31 males aged between 21 and 60 were invited using the stratified random sampling method for the audio recording of different emotions. Subsequently, parameter values of acoustic features were extracted using Praat voice software. Finally, parameter values were analyzed using a Two-way ANOVA, mixed-design analysis in SPSS software. Results show that gender and emotional states of the PAD model vary among seven major acoustic features. Moreover, their difference values and rankings also vary. The research conclusions lay a theoretical foundation for AI emotional voice interaction and solve deep learning's current dilemma in emotional recognition and parameter optimization of the emotional synthesis model due to the lack of explanatory power.

Analysis of Personality and EEG Features in Emotion Recognition Using Machine Learning Techniques to Classify Arousal and Valence Labels

We analyzed the contribution of electroencephalogram (EEG) data, age, sex, and personality traits to emotion recognition processes—through the classification of arousal, valence, and discrete emotions labels—using feature selection techniques and machine learning classifiers. EEG traits and age, sex, and personality traits were retrieved from a well-known dataset—AMIGOS—and two sets of traits were built to analyze the classification performance. We found that age, sex, and personality traits were not significantly associated with the classification of arousal, valence and discrete emotions using machine learning. The added EEG features increased the classification accuracies (compared with the original report), for arousal and valence labels. Classification of arousal and valence labels achieved higher than chance levels; however, they did not exceed 70% accuracy in the different tested scenarios. For discrete emotions, the mean accuracies and the mean area under the curve scores were higher than chance; however, F1 scores were low, implying that several false positives and false negatives were present. This study highlights the performance of EEG traits, age, sex, and personality traits using emotion classifiers. These findings could help to understand the traits relationship in a technological and data level for personalized human-computer interactions systems.

Effects of Expectancy on Cognitive Performance, Mood, and Psychophysiology in Healthy Adolescents and Their Parents in an Experimental Study

OBJECTIVE

Placebo effects on cognitive performance and mood and their underlying mechanisms have rarely been investigated in adolescents. Therefore, the following hypotheses were investigated with an experimental paradigm: (1) placebo effects could be larger in adolescents than in adults, (2) parents' expectations influence their adolescents' expectations and placebo effects, and (3) a decrease in stress levels could be an underlying mechanism of placebo effects.

METHODS

Twenty-six healthy adolescents (13.8 ± 1.6 years, 14 girls) each with a parent (45.5 ± 4.2 years, 17 mothers) took part in an experimental within-subjects study. On two occasions, a transdermal patch was applied to their hips and they received an envelope containing either the information that it is a Ginkgo patch to improve cognitive performance and mood, or it is an inactive placebo patch, in counterbalanced order. Cognitive performance and mood were assessed with a parametric Go/No-Go task (PGNG), a modification of California Verbal Learning Test, and Profile of Mood Scales (POMS). Subjects rated their expectations about Ginkgo's effects before patch application as well as their subjective assessment of its effects after the tests. An electrocardiogram and skin conductance levels (SCLs) were recorded and root mean square of successive differences (RMSSD), high-frequency power (HF), and the area under the curve of the SCL (AUC) were analyzed as psychophysiological stress markers.

RESULTS

Expectations did not differ between adolescents and parents and were correlated concerning reaction times only. Overall, expectations did not influence placebo effects. There was only one significant placebo effect on the percentage of correct inhibited trials in one level of the PGNG in adolescents, but not in parents. RMSSD and HF significantly increased, and AUC decreased from pre- to post-patch application in adolescents, but not in parents.

CONCLUSION

With this experimental paradigm, we could not induce relevant placebo effects in adolescents and parents. This could be due to aspects of the study design such as application form and substance, and that healthy subjects were employed. Nevertheless, we could show that adolescents are more sensitive to psychophysiological reactions related with interventions which could be part of the underlying mechanisms of placebo effects in adolescents.

Multimodal Depression Detection: Fusion of Electroencephalography and Paralinguistic Behaviors Using a Novel Strategy for Classifier Ensemble

Currently, depression has become a common mental disorder and one of the main causes of disability worldwide. Due to the difference in depressive symptoms evoked by individual differences, how to design comprehensive and effective depression detection methods has become an urgent demand. This study explored from physiological and behavioral perspectives simultaneously and fused pervasive electroencephalography (EEG) and vocal signals to make the detection of depression more objective, effective and convenient. After extraction of several effective features for these two types of signals, we trained six representational classifiers on each modality, then denoted diversity and correlation of decisions from different classifiers using co-decision tensor and combined these decisions into the ultimate classification result with multi-agent strategy. Experimental results on 170 (81 depressed patients and 89 normal controls) subjects showed that the proposed multi-modal depression detection strategy is superior to the single-modal classifiers or other typical late fusion strategies in accuracy, f1-score and sensitivity. This work indicates that late fusion of pervasive physiological and behavioral signals is promising for depression detection and the multi-agent strategy can take advantage of diversity and correlation of different classifiers effectively to gain a better final decision.

Positive and negative affect as predictors of social functioning in Spanish children

The aim of this study was to analyze the relationship between affect in its two commonly used theoretical categories, positive affect (PA) and negative affect (NA), and social functioning dimensions (school performance, family relationships, peer relationships and home duties/self-care). The sample comprised 390 students of primary education aged 8–11 years (M = 9.39; SD = 1.15). The short-form of the Positive and Negative Affect Schedule for children (PANAS-C-SF) and the Child and Adolescent Social Adaptive Functioning Scale (CASAFS) were used. Student’s t tests indicated that those reporting high levels on all the social functioning dimensions also reported significantly higher levels of PA than peers who reported low levels; by contrast, students reporting high levels of social functioning reported significantly lower levels of NA than peers who reported low levels. Similarly, logistic regression analyses showed that an increase in PA increased probability of high levels of social functioning, and that an increase in NA decreased the probability of presenting high levels of social functioning dimensions, with the exception of school performance. These results expand the PA and NA relationship with social functioning reported in adults to Spanish children, which is potentially of interest in the fields of Education and Psychology.