Extraction of facial features as indicators of stress and anxiety

A novel multi-modal depression detection approach based on mobile crowd sensing and task-based mechanisms

Depression has become a global concern, and COVID-19 also has caused a big surge in its incidence. Broadly, there are two primary methods of detecting depression: Task-based and Mobile Crowd Sensing (MCS) based methods. These two approaches, when integrated, can complement each other. This paper proposes a novel approach for depression detection that combines real-time MCS and task-based mechanisms. We aim to design an end-to-end machine learning pipeline, which involves multimodal data collection, feature extraction, feature selection, fusion, and classification to distinguish between depressed and non-depressed subjects. For this purpose, we created a real-world dataset of depressed and non-depressed subjects. We experimented with: various features from multi-modalities, feature selection techniques, fused features, and machine learning classifiers such as Logistic Regression, Support Vector Machines (SVM), etc. for classification. Our findings suggest that combining features from multiple modalities perform better than any single data modality, and the best classification accuracy is achieved when features from all three data modalities are fused. Feature selection method based on Pearson's correlation coefficients improved the accuracy in comparison with other methods. Also, SVM yielded the best accuracy of 86%. Our proposed approach was also applied on benchmarking dataset, and results demonstrated that the multimodal approach is advantageous in performance with state-of-the-art depression recognition techniques.

Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods.

Imagery of negative interpersonal experiences influence the neural mechanisms of social interaction

Negative interpersonal experiences are a key contributor to psychiatric disorders. While previous research has shown that negative interpersonal experiences influence social cognition, less is known about the effects on participation in social interactions and the underlying neurobiology. To address this, we developed a new naturalistic version of a gaze-contingent paradigm using real video sequences of gaze behaviour that respond to the participants' gaze in real-time in order to create a believable and continuous interactive social situation. Additionally, participants listened to two autobiographical audio-scripts that guided them to imagine a recent stressful and a relaxing situation and performed the gaze-based social interaction task before and after the presentation of either the stressful or the relaxing audio-script. Our results demonstrate that the social interaction task robustly recruits brain areas with known involvement in social cognition, namely the medial prefrontal cortex, bilateral temporoparietal junction, superior temporal sulcus as well as the precuneus. Imagery of negative interpersonal experiences compared to relaxing imagery led to a prolonged change in affective state and to increased brain responses during the subsequent social interaction paradigm in the temporoparietal junction, medial prefrontal cortex, anterior cingulate cortex, precuneus and inferior frontal gyrus. Taken together this study presents a new naturalistic social interaction paradigm suitable to study the neural mechanisms of social interaction and the results demonstrate that the imagery of negative interpersonal experiences affects social interaction on neural levels.

Facial Expression Recognition With Machine Learning and Assessment of Distress in Patients With Cancer

OBJECTIVES

To estimate the effectiveness of combining facial expression recognition and machine learning for better detection of distress.

SAMPLE & SETTING

232 patients with cancer in Sichuan University West China Hospital in Chengdu, China.

METHODS & VARIABLES

The Distress Thermometer (DT) and Hospital Anxiety and Depression Scale (HADS) were used as instruments. The HADS included scores for anxiety (HADS-A), depression (HADS-D), and total score (HADS-T). Distressed patients were defined by the DT cutoff score of 4, the HADS-A cutoff score of 8 or 9, the HADS-D cutoff score of 8 or 9, or the HADS-T cutoff score of 14 or 15. The authors applied histogram of oriented gradients to extract facial expression features from face images, and used a support vector machine as the classifier.

RESULTS

The facial expression features showed feasible differentiation ability on cases classified by DT and HADS.

IMPLICATIONS FOR NURSING

Facial expression recognition could serve as a supplementary screening tool for improving the accuracy of distress assessment and guide strategies for treatment and nursing.

Video-Based Stress Detection through Deep Learning

Stress has become an increasingly serious problem in the current society, threatening mankind’s well-beings. With the ubiquitous deployment of video cameras in surroundings, detecting stress based on the contact-free camera sensors becomes a cost-effective and mass-reaching way without interference of artificial traits and factors. In this study, we leverage users’ facial expressions and action motions in the video and present a two-leveled stress detection network (TSDNet). TSDNet firstly learns face- and action-level representations separately, and then fuses the results through a stream weighted integrator with local and global attention for stress identification. To evaluate the performance of TSDNet, we constructed a video dataset containing 2092 labeled video clips, and the experimental results on the built dataset show that: (1) TSDNet outperformed the hand-crafted feature engineering approaches with detection accuracy 85.42% and F1-Score 85.28%, demonstrating the feasibility and effectiveness of using deep learning to analyze one’s face and action motions; and (2) considering both facial expressions and action motions could improve detection accuracy and F1-Score of that considering only face or action method by over 7%.

The impact of perceived stress on skin ageing

Skin ageing can be divided according to phenotypical features into intrinsic (by the passage of time) and extrinsic (with the addition of the effects of environmental factors). Photoageing is by far the most researched factor of extrinsic ageing but the additional impact of other factors such as cigarette smoking and exposure to air pollution ought to be taken into account. One of the least researched topics in relation to extrinsic skin ageing is the impact of psychological stress. A contemporary review of response of human skin to stress describes the molecular mechanisms of extrinsic skin ageing, but has fallen short of explaining resilience to stress exhibited by people. Mechanisms to regulate gene expression, define cellular identity and promote functionality are responsible for the adaptive response to stressful events. Conversely, maladaptive response of human tissues to chronic stress appears to have an impact on gene regulation. Epigenetics is the study of heritable changes in organisms due to modifications in gene activity and expression, as opposed to the genetic code (DNA genome). Chronic stress appears to be an important factor in determining an individual's vulnerability to ageing and age-related comorbidities via epigenetic modifications. Forerunners in epigenetic research recognized the necessity of a reliable biomarker in order to develop a better understanding of the role of epigenomics in ageing. Genomic DNA methylation patterns (DNAm) appear to be valuable in age prediction but variability in specificity exists across species of mammals, human races and tissues. Neuroscience research appears to be leading the way in epigenomics whilst the lack of a valid and reliable DNAm-associated age predictor compatible with human skin tissue hinders research endeavours for the epigenetics of skin ageing.

Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System

We present the first study in the literature that has aimed to determine Depression Anxiety Stress Scale (DASS) levels by analyzing facial expressions using Facial Action Coding System (FACS) by means of a unique noninvasive architecture on three layers designed to offer high accuracy and fast convergence: in the first layer, Active Appearance Models (AAM) and a set of multiclass Support Vector Machines (SVM) are used for Action Unit (AU) classification; in the second layer, a matrix is built containing the AUs' intensity levels; and in the third layer, an optimal feedforward neural network (FFNN) analyzes the matrix from the second layer in a pattern recognition task, predicting the DASS levels. We obtained 87.2% accuracy for depression, 77.9% for anxiety, and 90.2% for stress. The average prediction time was 64 s, and the architecture could be used in real time, allowing health practitioners to evaluate the evolution of DASS levels over time. The architecture could discriminate with 93% accuracy between healthy subjects and those affected by Major Depressive Disorder (MDD) or Post-traumatic Stress Disorder (PTSD), and 85% for Generalized Anxiety Disorder (GAD). For the first time in the literature, we determined a set of correlations between DASS, induced emotions, and FACS, which led to an increase in accuracy of 5%. When tested on AVEC 2014 and ANUStressDB, the method offered 5% higher accuracy, sensitivity, and specificity compared to other state-of-the-art methods.