Deep convolution network based emotion analysis towards mental health care

Sensing emotional valence and arousal dynamics through automated facial action unit analysis

Information about the concordance between dynamic emotional experiences and objective signals is practically useful. Previous studies have shown that valence dynamics can be estimated by recording electrical activity from the muscles in the brows and cheeks. However, whether facial actions based on video data and analyzed without electrodes can be used for sensing emotion dynamics remains unknown. We investigated this issue by recording video of participants' faces and obtaining dynamic valence and arousal ratings while they observed emotional films. Action units (AUs) 04 (i.e., brow lowering) and 12 (i.e., lip-corner pulling), detected through an automated analysis of the video data, were negatively and positively correlated with dynamic ratings of subjective valence, respectively. Several other AUs were also correlated with dynamic valence or arousal ratings. Random forest regression modeling, interpreted using the SHapley Additive exPlanation tool, revealed non-linear associations between the AUs and dynamic ratings of valence or arousal. These results suggest that an automated analysis of facial expression video data can be used to estimate dynamic emotional states, which could be applied in various fields including mental health diagnosis, security monitoring, and education.

Discriminant Input Processing Scheme for Self-Assisted Intelligent Healthcare Systems

Modern technology and analysis of emotions play a crucial role in enabling intelligent healthcare systems to provide diagnostics and self-assistance services based on observation. However, precise data predictions and computational models are critical for these systems to perform their jobs effectively. Traditionally, healthcare monitoring has been the primary emphasis. However, there were a couple of negatives, including the pattern feature generating the method's scalability and reliability, which was tested with different data sources. This paper delves into the Discriminant Input Processing Scheme (DIPS), a crucial instrument for resolving challenges. Data-segmentation-based complex processing techniques allow DIPS to merge many emotion analysis streams. The DIPS recommendation engine uses segmented data characteristics to sift through inputs from the emotion stream for patterns. The recommendation is more accurate and flexible since DIPS uses transfer learning to identify similar data across different streams. With transfer learning, this study can be sure that the previous recommendations and data properties will be available in future data streams, making the most of them. Data utilization ratio, approximation, accuracy, and false rate are some of the metrics used to assess the effectiveness of the advised approach. Self-assisted intelligent healthcare systems that use emotion-based analysis and state-of-the-art technology are crucial when managing healthcare. This study improves healthcare management's accuracy and efficiency using computational models like DIPS to guarantee accurate data forecasts and recommendations.

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

Mental illness has grown to become a prevalent and global health concern that affects individuals across various demographics. Timely detection and accurate diagnosis of mental disorders are crucial for effective treatment and support as late diagnosis could result in suicidal, harmful behaviors and ultimately death. To this end, the present study introduces a novel pipeline for the analysis of facial expressions, leveraging both the AffectNet and 2013 Facial Emotion Recognition (FER) datasets. Consequently, this research goes beyond traditional diagnostic methods by contributing a system capable of generating a comprehensive mental disorder dataset and concurrently predicting mental disorders based on facial emotional cues. Particularly, we introduce a hybrid architecture for mental disorder detection leveraging the state-of-the-art object detection algorithm, YOLOv8 to detect and classify visual cues associated with specific mental disorders. To achieve accurate predictions, an integrated learning architecture based on the fusion of Convolution Neural Networks (CNNs) and Visual Transformer (ViT) models is developed to form an ensemble classifier that predicts the presence of mental illness (e.g., depression, anxiety, and other mental disorder). The overall accuracy is improved to about 81% using the proposed ensemble technique. To ensure transparency and interpretability, we integrate techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) and saliency maps to highlight the regions in the input image that significantly contribute to the model's predictions thus providing healthcare professionals with a clear understanding of the features influencing the system's decisions thereby enhancing trust and more informed diagnostic process.

Emotion Recognition Using Hierarchical Spatiotemporal Electroencephalogram Information from Local to Global Brain Regions

To understand human emotional states, local activities in various regions of the cerebral cortex and the interactions among different brain regions must be considered. This paper proposes a hierarchical emotional context feature learning model that improves multichannel electroencephalography (EEG)-based emotion recognition by learning spatiotemporal EEG features from a local brain region to a global brain region. The proposed method comprises a regional brain-level encoding module, a global brain-level encoding module, and a classifier. First, multichannel EEG signals grouped into nine regions based on the functional role of the brain are input into a regional brain-level encoding module to learn local spatiotemporal information. Subsequently, the global brain-level encoding module improved emotional classification performance by integrating local spatiotemporal information from various brain regions to learn the global context features of brain regions related to emotions. Next, we applied a two-layer bidirectional gated recurrent unit (BGRU) with self-attention to the regional brain-level module and a one-layer BGRU with self-attention to the global brain-level module. Experiments were conducted using three datasets to evaluate the EEG-based emotion recognition performance of the proposed method. The results proved that the proposed method achieves superior performance by reflecting the characteristics of multichannel EEG signals better than state-of-the-art methods.

Mathematical model and artificial intelligence for diagnosis of Alzheimer's disease

Degeneration of the neurological system linked to cognitive deficits, daily living exercise clutters, and behavioral disturbing impacts may define Alzheimer's disease. Alzheimer's disease research conducted later in life focuses on describing ways for early detection of dementia, a kind of mental disorder. To tailor our care to each patient, we utilized visual cues to determine how they were feeling. We did this by outlining two approaches to diagnosing a person's mental health. Support vector machine is the first technique. Image characteristics are extracted using a fractal model for classification in this method. With this technique, the histogram of a picture is modeled after a Gaussian distribution. Classification was performed with several support vector machines kernels, and the outcomes were compared. Step two proposes using a deep convolutional neural network architecture to identify Alzheimer's-related mental disorders. According to the findings, the support vector machines approach accurately recognized over 93% of the photos tested. The deep convolutional neural network approach was one hundred percent accurate during model training, whereas the support vector machines approach achieved just 93 percent accuracy. In contrast to support vector machines accuracy of 89.3%, the deep convolutional neural network model test findings were accurate 98.8% of the time. Based on the findings reported here, the proposed deep convolutional neural network architecture may be used for diagnostic purposes involving the patient's mental state.

A feature boosted deep learning method for automatic facial expression recognition

Automatic facial expression recognition (FER) plays a crucial role in human-computer based applications such as psychiatric treatment, classroom assessment, surveillance systems, and many others. However, automatic FER is challenging in real-time environment. The traditional methods used handcrafted methods for FER but mostly failed to produce superior results in the wild environment. In this regard, a deep learning-based FER approach with minimal parameters is proposed, which gives better results for lab-controlled and wild datasets. The method uses features boosting module with skip connections which help to focus on expression-specific features. The proposed approach is applied to FER-2013 (wild dataset), JAFFE (lab-controlled), and CK+ (lab-controlled) datasets which achieve accuracy of 70.21%, 96.16%, and 96.52%. The observed experimental results demonstrate that the proposed method outperforms the other related research concerning accuracy and time.

An Intelligent Mental Health Identification Method for College Students: A Mixed-Method Study

PURPOSE

Mental health assessments that combine patients' facial expressions and behaviors have been proven effective, but screening large-scale student populations for mental health problems is time-consuming and labor-intensive. This study aims to provide an efficient and accurate intelligent method for further psychological diagnosis and treatment, which combines artificial intelligence technologies to assist in evaluating the mental health problems of college students.

MATERIALS AND METHODS

We propose a mixed-method study of mental health assessment that combines psychological questionnaires with facial emotion analysis to comprehensively evaluate the mental health of students on a large scale. The Depression Anxiety and Stress Scale-21(DASS-21) is used for the psychological questionnaire. The facial emotion recognition model is implemented by transfer learning based on neural networks, and the model is pre-trained using FER2013 and CFEE datasets. Among them, the FER2013 dataset consists of 48 × 48-pixel face gray images, a total of 35,887 face images. The CFEE dataset contains 950,000 facial images with annotated action units (au). Using a random sampling strategy, we sent online questionnaires to 400 college students and received 374 responses, and the response rate was 93.5%. After pre-processing, 350 results were available, including 187 male and 153 female students. First, the facial emotion data of students were collected in an online questionnaire test. Then, a pre-trained model was used for emotion recognition. Finally, the online psychological questionnaire scores and the facial emotion recognition model scores were collated to give a comprehensive psychological evaluation score.

RESULTS

The experimental results of the facial emotion recognition model proposed to show that its classification results are broadly consistent with the mental health survey results. This model can be used to improve efficiency. In particular, the accuracy of the facial emotion recognition model proposed in this paper is higher than that of the general mental health model, which only uses the traditional single questionnaire. Furthermore, the absolute errors of this study in the three symptoms of depression, anxiety, and stress are lower than other mental health survey results and are only 0.8%, 8.1%, 3.5%, and 1.8%, respectively.

CONCLUSION

The mixed method combining intelligent methods and scales for mental health assessment has high recognition accuracy. Therefore, it can support efficient large-scale screening of students' psychological problems.

Analysis model of college students' mental health based on online community topic mining and emotion analysis in novel coronavirus epidemic situation

Under the epidemic situation of COVID-19, university students have different levels of anxiety, depression, and other psychological problems, and these differing levels present different challenges. Therefore, universities and relevant departments should carry out accurate psychological health education for university students. Through research, this paper found that students' psychological problems during the COVID-19 epidemic were mainly reflected in four aspects: depression, interpersonal relationship, sleep and eating disorders, and compulsive behavior. Through the discussion of family of origin, self-awareness and motivation attribution, and social pressure, this paper analyzed the causes of psychological problems. The information resources of the network are usually unstructured data, and the text information, as the most typical unstructured data, occupies a large proportion. Moreover, this text information often contains users' emotional response to major events. In this paper, a data preprocessing system is designed, and three data preprocessing rules are defined: expression data conversion rules, data deduplication rules and invalid data cleaning rules. The characteristics of online community text data are analyzed, and the text feature extraction method is selected according to its characteristics. The results of this study show that the proportion of university students with psychological problems is about 23%, which is slightly higher than the research results during the non-epidemic period. This paper suggests that college students should master methods of self-regulation, improve their levels of physical exercise, improve their physical fitness, and establish and improve their defense mechanisms to alleviate psychological conflicts and pressures.

An Assessment and Analysis Model of Psychological Health of College Students Based on Convolutional Neural Networks

Psychological health assessment and psychological problem identification essentially belong to problems of pattern recognition or nonlinear classification; its system contains complex nonlinear interactions among various factors, having basic characteristics of multivariable, multilevel, and strong coupling. An important problem in the field of artificial intelligence solved by convolutional neural networks (CNN) is to simplify complex problems, minimize the number of parameters, and thus greatly improve the algorithm's performance. Therefore, CNN has outstanding advantages in establishing the assessment and analysis model of college students' psychological health. This study determined the psychological health standards of college students, selected measurement tools for college students' psychological state, elaborated the principles of psychological assessment based on text information, performed the sample set data establishment and data processing of the assessment and analysis model of psychological health, conducted network establishment, training, and simulation, carried out a case experiment and its result analysis, explored the cause analysis of college students' psychological health problems, and finally discussed the prevention and intervention of college students' psychological problems. The study results show that the input and output of the CNN-based assessment and analysis model of college students' psychological health are their evaluation data and assessment results, respectively, and the optimal hyperparameters of the model are determined through fold cross-validation analysis to improve the model's over-fitting problem. After the training is completed, the model can predict the changes in college students' psychological state in the future through the psychological test data. The CNN uses supervised machine learning method to construct an assessment and analysis model of college students' psychological health, and establishes the mapping relationship between college students' personal background and their psychological health. The network error continuously adjusts network connection weight according to gradient descent algorithm to minimize its error, so that the convolutional layer and the pooling layer can learn the optimized feature expression of the input data.

A Hybrid Model for Driver Emotion Detection Using Feature Fusion Approach

Machine and deep learning techniques are two branches of artificial intelligence that have proven very efficient in solving advanced human problems. The automotive industry is currently using this technology to support drivers with advanced driver assistance systems. These systems can assist various functions for proper driving and estimate drivers' capability of stable driving behavior and road safety. Many studies have proved that the driver's emotions are the significant factors that manage the driver's behavior, leading to severe vehicle collisions. Therefore, continuous monitoring of drivers' emotions can help predict their behavior to avoid accidents. A novel hybrid network architecture using a deep neural network and support vector machine has been developed to predict between six and seven driver's emotions in different poses, occlusions, and illumination conditions to achieve this goal. To determine the emotions, a fusion of Gabor and LBP features has been utilized to find the features and been classified using a support vector machine classifier combined with a convolutional neural network. Our proposed model achieved better performance accuracy of 84.41%, 95.05%, 98.57%, and 98.64% for FER 2013, CK+, KDEF, and KMU-FED datasets, respectively.

Automated Computer Vision Assessment of Hypomimia in Parkinson Disease: Proof-of-Principle Pilot Study

BACKGROUND

Facial expressions require the complex coordination of 43 different facial muscles. Parkinson disease (PD) affects facial musculature leading to "hypomimia" or "masked facies."

OBJECTIVE

We aimed to determine whether modern computer vision techniques can be applied to detect masked facies and quantify drug states in PD.

METHODS

We trained a convolutional neural network on images extracted from videos of 107 self-identified people with PD, along with 1595 videos of controls, in order to detect PD hypomimia cues. This trained model was applied to clinical interviews of 35 PD patients in their on and off drug motor states, and seven journalist interviews of the actor Alan Alda obtained before and after he was diagnosed with PD.

RESULTS

The algorithm achieved a test set area under the receiver operating characteristic curve of 0.71 on 54 subjects to detect PD hypomimia, compared to a value of 0.75 for trained neurologists using the United Parkinson Disease Rating Scale-III Facial Expression score. Additionally, the model accuracy to classify the on and off drug states in the clinical samples was 63% (22/35), in contrast to an accuracy of 46% (16/35) when using clinical rater scores. Finally, each of Alan Alda's seven interviews were successfully classified as occurring before (versus after) his diagnosis, with 100% accuracy (7/7).

CONCLUSIONS

This proof-of-principle pilot study demonstrated that computer vision holds promise as a valuable tool for PD hypomimia and for monitoring a patient's motor state in an objective and noninvasive way, particularly given the increasing importance of telemedicine.

COVIDetectioNet: COVID-19 diagnosis system based on X-ray images using features selected from pre-learned deep features ensemble

The recent novel coronavirus (also known as COVID-19) has rapidly spread worldwide, causing an infectious respiratory disease that has killed hundreds of thousands and infected millions. While test kits are used for diagnosis of the disease, the process takes time and the test kits are limited in their availability. However, the COVID-19 disease is also diagnosable using radiological images taken through lung X-rays. This process is known to be both faster and more reliable as a form of identification and diagnosis. In this regard, the current study proposes an expert-designed system called COVIDetectioNet model, which utilizes features selected from combination of deep features for diagnosis of COVID-19. For this purpose, a pretrained Convolutional Neural Network (CNN)-based AlexNet architecture that employed the transfer learning approach, was used. The effective features that were selected using the Relief feature selection algorithm from all layers of the architecture were then classified using the Support Vector Machine (SVM) method. To verify the validity of the model proposed, a total of 6092 X-ray images, classified as Normal (healthy), COVID-19, and Pneumonia, were obtained from a combination of public datasets. In the experimental results, an accuracy of 99.18% was achieved using the model proposed. The results demonstrate that the proposed COVIDetectioNet model achieved a superior level of success when compared to previous studies.

Smart Sensor Based on Biofeedback to Measure Child Relaxation in Out-of-Home Care

Children from out-of-home care are a vulnerable population that faces high stress and anxiety levels due to stressful experiences, such as being abused, being raped, and violence. This problem could have negative effects on their bio-psycho-social well-being if they are not provided with comprehensive psychological treatment. Numerous methods have been developed to help them relax, but there are no current approaches for assessing the relaxation level they reach. Based on this, a novel smart sensor that can evaluate the level of relaxation a child experiences is developed in this paper. It evaluates changes in thermal biomarkers (forehead, right and left cheek, chin, and maxillary) and heart rate (HR). Then, through a k-nearest neighbors (K-NN) intelligent classifier, four possible levels of relaxation can be obtained: no-relax, low-relax, relax, and very-relax. Additionally, an application (called i-CARE) for anxiety management, which is based on biofeedback diaphragmatic breathing, guided imagery, and video games, is evaluated. After testing the developed smart sensor, an 89.7% accuracy is obtained. The smart sensor used provides a reliable measurement of relaxation levels and the i-CARE application is effective for anxiety management, both of which are focused on children exposed to out-of-home care conditions.