From Emotions to Mood Disorders: A Survey on Gait Analysis Methodology

An Ensemble Classification Model for Depression Based on Wearable Device Sleep Data

Depression is one of the most common mental disorders, with sleep disturbances as typical symptoms. With the popularity of wearable devices increasing in recent years, more and more people wear portable devices to track sleep quality. Based on this, we believe that depression detection through wearable sleep data is more intelligent and economical. However, the majority of wearable devices face the problem of missing data during the data collection process. Otherwise, most existing studies of depression identification focus on the utilization of complex data, making it difficult to generalize and susceptible to noise interference. To address these issues, we propose a systematic ensemble classification model for depression (ECD). For the missing data problem of wearable devices, we design an improved GAIN method to further control the generation range of interpolated values, which can achieve a more reasonable treatment of missing values. Compared with the original GAIN approach, the improved method shows a 28.56% improvement when using MAE as the metric. For depression recognition, we use ensemble learning to construct a depression classification model which combines five classification models, including SVM, KNN, LR, CBR, and DT. Ensemble learning can improve the model's robustness and generalization. The voting mechanism is used in several places to improve noise immunity. The final classification model performed great on the dataset, with a precision of 92.55% and a recall of 91.89%. These results illustrate how efficient this method is in automatically detecting depression.

EPIC: Emotion Perception by Spatio-Temporal Interaction Context of Gait

Recently, psychophysiological computing has received considerable attention. Due to easy acquisition at a distance and less conscious initiation, gait-based emotion recognition is considered as a valuable research branch in the field of psychophysiological computing. However, most existing methods rarely explore the spatio-temporal context of gait, which limits the ability to capture the higher-order relationship between emotion and gait. In this paper, we utilize a range of research, including psychophysiological computing and artificial intelligence, to propose an integrated emotion perception framework called EPIC, which can find novel joint topology and generate thousands of synthetic gaits by spatio-temporal interaction context. First, we analyze the joint coupling among non-adjacent joints by calculating Phase Lag Index (PLI), which can discover the latent connection among body joints. Second, to synthesize more sophisticated and accurate gait sequences, we explore the effect of spatio-temporal constraints, and propose a new loss function that utilizes the Dynamic Time Warping (DTW) algorithm and pseudo-velocity curve to constrain the output of Gated Recurrent Units (GRU). Finally, Spatial Temporal Graph Convolution Networks (ST-GCN) is used to classify emotions using the generation and the real data. Experimental results demonstrate our approach achieves the accuracy of 89.66%, and outperforms the state-of-the-art methods on Emotion-Gait dataset.

Circadian rhythms of melatonin and its relationship with anhedonia in patients with mood disorders: a cross-sectional study

BACKGROUND

Mood disorders are strongly associated with melatonin disturbances. However, it is unclear whether there is a difference in melatonin concentrations and melatonin circadian rhythm profiles between depression and bipolar disorder. In addition, the relationship between anhedonia, a common symptom of affective disorders, and its melatonin circadian rhythm remains under-investigated.

METHODS

Thirty-four patients with depression disorder, 20 patients diagnosed with bipolar disorder and 21 healthy controls participated in this study. The Revised Physical Anhedonia Scale (RPAS) was performed to assess anhedonia. Saliva samples were collected from all subjects at fixed time points (a total of 14 points) in two consecutive days for measuring the melatonin concentrations to fit circadian rhythms of subjects. Melatonin circadian rhythms were compared between the three groups using ANOVA. Partial correlation analysis and linear regression analysis were used to explore the correlation between melatonin rhythm variables and anhedonia.

RESULTS

We found that the peak phase of melatonin in the depression group was significantly advanced compared to the control group (P < 0.001) and the bipolar disorder group (P = 0.004). The peak phase of melatonin and RPAS showed a negative correlation (P = 0.003) in depression patients, which was also demonstrated in the multiple linear regression model (B=-2.47, P = 0.006).

CONCLUSIONS

These results suggest that circadian rhythms of melatonin are differentiated in depression and bipolar disorder and correlate with anhedonia in depression. Future research needs to explore the neurobiological mechanisms linking anhedonia and melatonin circadian rhythms in depressed patients.

Assessment of the human response to acute mental stress-An overview and a multimodal study

Numerous vital signs are reported in association with stress response assessment, but their application varies widely. This work provides an overview over methods for stress induction and strain assessment, and presents a multimodal experimental study to identify the most important vital signs for effective assessment of the response to acute mental stress. We induced acute mental stress in 65 healthy participants with the Mannheim Multicomponent Stress Test and acquired self-assessment measures (Likert scale, Self-Assessment Manikin), salivary α-amylase and cortisol concentrations as well as 60 vital signs from biosignals, such as heart rate variability parameters, QT variability parameters, skin conductance level, and breath rate. By means of statistical testing and a self-optimizing logistic regression, we identified the most important biosignal vital signs. Fifteen biosignal vital signs related to ventricular repolarization variability, blood pressure, skin conductance, and respiration showed significant results. The logistic regression converged with QT variability index, left ventricular work index, earlobe pulse arrival time, skin conductance level, rise time and number of skin conductance responses, breath rate, and breath rate variability (F1 = 0.82). Self-assessment measures indicated successful stress induction. α-amylase and cortisol showed effect sizes of -0.78 and 0.55, respectively. In summary, the hypothalamic-pituitary-adrenocortical axis and sympathetic nervous system were successfully activated. Our findings facilitate a coherent and integrative understanding of the assessment of the stress response and help to align applications and future research concerning acute mental stress.

Emotional states affect steady state walking performance

Gait is a large component and indicator of health. Many factors affect gait including age, disease, and even mood disorders. Few studies have looked at the influence of emotional states on gait. This study aimed to investigate the influence of emotional states on walking performance to understand whether an emotional state may be an important factor to consider when evaluating gait. Thirty-six young adults were recruited (23F, 13M) and performed a neutral baseline condition of walking which included six passes of walking across an 8m walkway (a total of 48m of walking). Participants then completed 6 pseudo-randomized emotional state induction conditions while immersive 360-degree videos were used to induce the following emotional conditions: happiness, excitement, sadness, fear, and anger. Participants viewed the emotion elicitation videos using a virtual reality head-mounted display (HMD), then rated their emotional state using self-assessment manikins and walked (without the HMD) over a pressure sensor walkway. One-way repeated measures ANOVA and pairwise comparisons were used to examine differences in gait parameters across the emotional conditions. Participants walked with significantly reduced step length and speed during the sadness condition compared to the other emotional conditions and the neutral condition. Furthermore, participants adjusted the timing of their walking during the sadness condition and walked with significantly increased step, stance, and swing times compared to other emotional conditions, but not the neutral condition. Step time was significantly reduced during the conditions of excitement and fear compared to the neutral condition. Emotions may impact variety of gait parameters involving pace and rhythm, however have little influence on gait variability and postural control. These results indicate that perhaps the emotions of sadness and excitement should be taken into account as potential confounds for future gait analysis.

Predicting the fatigue in Parkinson's disease using inertial sensor gait data and clinical characteristics

Objectives

The study aimed to analyze the clinical features and gait characteristics of patients with Parkinson's disease (PD) who also suffer from fatigue and to develop a model that can help identify fatigue states in the early stages of PD.

Methodology

A total of 81 PD patients have been enrolled for the Parkinson's Fatigue Scale (PFS-16) assessment and divided into two groups: patients with or without fatigue. Neuropsychological assessments of the two groups, including motor and non-motor symptoms, were collected. The patient's gait characteristics were collected using a wearable inertial sensor device.

Results

PD patients who experienced fatigue had a more significant impairment of motor symptoms than those who did not, and the experience of fatigue became more pronounced as the disease progressed. Patients with fatigue had more significant mood disorders and sleep disturbances, which can lead to a poorer quality of life. PD patients with fatigue had shorter step lengths, lower velocity, and stride length and increased stride length variability. As for kinematic parameters, PD patients with fatigue had lower shank-forward swing max, trunk-max sagittal angular velocity, and lumbar-max coronal angular velocity than PD patients without fatigue. The binary logistic analysis found that Movement Disorder Society-Unified Parkinson's Disease Rating Scale-I (MDS-UPDRS-I) scores, Hamilton Depression Scale (HAMD) scores, and stride length variability independently predicted fatigue in PD patients. The area under the curve (AUC) of these selected factors in the receiver operating characteristic (ROC) analysis was 0.900. Moreover, HAMD might completely mediate the association between Hamilton Anxiety Scale (HAMA) scores and fatigue (indirect effect: β = 0.032, 95% confidence interval: 0.001-0.062), with a percentage of mediation of 55.46%.

Conclusion

Combining clinical characteristics and gait cycle parameters, including MDS-UPDRS-I scores, HAMD scores, and stride length variability, can identify PD patients with a high fatigue risk.

Person Recognition Based on Deep Gait: A Survey

Gait recognition, also known as walking pattern recognition, has expressed deep interest in the computer vision and biometrics community due to its potential to identify individuals from a distance. It has attracted increasing attention due to its potential applications and non-invasive nature. Since 2014, deep learning approaches have shown promising results in gait recognition by automatically extracting features. However, recognizing gait accurately is challenging due to the covariate factors, complexity and variability of environments, and human body representations. This paper provides a comprehensive overview of the advancements made in this field along with the challenges and limitations associated with deep learning methods. For that, it initially examines the various gait datasets used in the literature review and analyzes the performance of state-of-the-art techniques. After that, a taxonomy of deep learning methods is presented to characterize and organize the research landscape in this field. Furthermore, the taxonomy highlights the basic limitations of deep learning methods in the context of gait recognition. The paper is concluded by focusing on the present challenges and suggesting several research directions to improve the performance of gait recognition in the future.

A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton-Raphson-Based Feature Selection Approach for Human Gait Recognition

The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models-MobilenetV2 and ShuffleNet-are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton-Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time.

Deep Gait Recognition: A Survey

Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

Associations Between Depression Symptom Severity and Daily-Life Gait Characteristics Derived From Long-Term Acceleration Signals in Real-World Settings: Retrospective Analysis

BACKGROUND

Gait is an essential manifestation of depression. However, the gait characteristics of daily walking and their relationships with depression have yet to be fully explored.

OBJECTIVE

The aim of this study was to explore associations between depression symptom severity and daily-life gait characteristics derived from acceleration signals in real-world settings.

METHODS

We used two ambulatory data sets (N=71 and N=215) with acceleration signals collected by wearable devices and mobile phones, respectively. We extracted 12 daily-life gait features to describe the distribution and variance of gait cadence and force over a long-term period. Spearman coefficients and linear mixed-effects models were used to explore the associations between daily-life gait features and depression symptom severity measured by the 15-item Geriatric Depression Scale (GDS-15) and 8-item Patient Health Questionnaire (PHQ-8) self-reported questionnaires. The likelihood-ratio (LR) test was used to test whether daily-life gait features could provide additional information relative to the laboratory gait features.

RESULTS

Higher depression symptom severity was significantly associated with lower gait cadence of high-performance walking (segments with faster walking speed) over a long-term period in both data sets. The linear regression model with long-term daily-life gait features (R2=0.30) fitted depression scores significantly better (LR test P=.001) than the model with only laboratory gait features (R2=0.06).

CONCLUSIONS

This study indicated that the significant links between daily-life walking characteristics and depression symptom severity could be captured by both wearable devices and mobile phones. The daily-life gait patterns could provide additional information for predicting depression symptom severity relative to laboratory walking. These findings may contribute to developing clinical tools to remotely monitor mental health in real-world settings.

Detection and assessment of Parkinson's disease based on gait analysis: A survey

Neurological disorders represent one of the leading causes of disability and mortality in the world. Parkinson's Disease (PD), for example, affecting millions of people worldwide is often manifested as impaired posture and gait. These impairments have been used as a clinical sign for the early detection of PD, as well as an objective index for pervasive monitoring of the PD patients in daily life. This review presents the evidence that demonstrates the relationship between human gait and PD, and illustrates the role of different gait analysis systems based on vision or wearable sensors. It also provides a comprehensive overview of the available automatic recognition systems for the detection and management of PD. The intervening measures for improving gait performance are summarized, in which the smart devices for gait intervention are emphasized. Finally, this review highlights some of the new opportunities in detecting, monitoring, and treating of PD based on gait, which could facilitate the development of objective gait-based biomarkers for personalized support and treatment of PD.

The Avatar's Gist: How to Transfer Affective Components From Dynamic Walking to Static Body Postures

Dynamic virtual representations of the human being can communicate a broad range of affective states through body movements, thus effectively studying emotion perception. However, the possibility of modeling static body postures preserving affective information is still fundamental in a broad spectrum of experimental settings exploring time-locked cognitive processes. We propose a novel automatic method for creating virtual affective body postures starting from kinematics data. Exploiting body features related to postural cues and movement velocity, we transferred the affective components from dynamic walking to static body postures of male and female virtual avatars. Results of two online experiments showed that participants coherently judged different valence and arousal levels in the avatar's body posture, highlighting the reliability of the proposed methodology. In addition, esthetic and postural cues made women more emotionally expressive than men. Overall, we provided a valid methodology to create affective body postures of virtual avatars, which can be used within different virtual scenarios to understand better the way we perceive the affective state of others.

Human Gait Analysis in Neurodegenerative Diseases: a Review

This paper reviews the recent literature on technologies and methodologies for quantitative human gait analysis in the context of neurodegnerative diseases. The use of technological instruments can be of great support in both clinical diagnosis and severity assessment of these pathologies. In this paper, sensors, features and processing methodologies have been reviewed in order to provide a highly consistent work that explores the issues related to gait analysis. First, the phases of the human gait cycle are briefly explained, along with some non-normal gait patterns (gait abnormalities) typical of some neurodegenerative diseases. The work continues with a survey on the publicly available datasets principally used for comparing results. Then the paper reports the most common processing techniques for both feature selection and extraction and for classification and clustering. Finally, a conclusive discussion on current open problems and future directions is outlined.

Comparison of predicted psychological distress among workers between artificial intelligence and psychiatrists: a cross-sectional study in Tsukuba Science City, Japan

OBJECTIVES

Psychological distress is a worldwide problem and a serious problem that needs to be addressed in the field of occupational health. This study aimed to use artificial intelligence (AI) to predict psychological distress among workers using sociodemographic, lifestyle and sleep factors, not subjective information such as mood and emotion, and to examine the performance of the AI models through a comparison with psychiatrists.

DESIGN

Cross-sectional study.

SETTING

We conducted a survey on psychological distress and living conditions among workers. An AI model for predicting psychological distress was created and then the results were compared in terms of accuracy with predictions made by psychiatrists.

PARTICIPANTS

An AI model of the neural network and six psychiatrists.

PRIMARY OUTCOME

The accuracies of the AI model and psychiatrists for predicting psychological distress.

METHODS

In total, data from 7251 workers were analysed to predict moderate and severe psychological distress. An AI model of the neural network was created and accuracy, sensitivity and specificity were calculated. Six psychiatrists used the same data as the AI model to predict psychological distress and conduct a comparison with the AI model.

RESULTS

The accuracies of the AI model and psychiatrists for predicting moderate psychological distress were 65.2% and 64.4%, respectively, showing no significant difference. The accuracies of the AI model and psychiatrists for predicting severe psychological distress were 89.9% and 85.5%, respectively, indicating that the AI model had significantly higher accuracy.

CONCLUSIONS

A machine learning model was successfully developed to screen workers with depressed mood. The explanatory variables used for the predictions did not directly ask about mood. Therefore, this newly developed model appears to be able to predict psychological distress among workers easily, regardless of their subjective views.

Magnetic resonance imaging in mood disorders: a bibliometric analysis from 1999 to 2020

Objective

Globally, mood disorders are highly prevalent, and are associated with increased morbidity and mortalities. Magnetic resonance imaging is widely used in the study of mood disorders. However, bibliometric analyses of the state of this field are lacking.

Methods

A literature search in the web of science core collection (WoSCC) for the period between 1945 and 2020 returned 3073 results. Data extracted from these publications include, publication year, journal names, countries of origin, institutions, author names and research areas. The bibliometric method, CiteSpace V and key words analysis were used to visualize the collaboration network and identify research trends, respectively.

Results

Since it was first reported in 1999, the use of magnetic resonance imaging in studies on mood disorders has been increasing. Biological psychiatry is the core journal that has extensively published on this topic, while the UNIV PITTSBURGH, USA, has the highest published papers on this topic. Keyword analysis indicated that studies on depression, bipolar disorders, and schizophrenia, with a focus on specific brain regions, including amygdala, prefrontal cortex and anterior cingulate cortex are key research topics.

Conclusion

Brain structure and network, sex differences, and treatment-associated brain changes are key topics of future research.

Cross-Subject and Cross-Modal Transfer for Generalized Abnormal Gait Pattern Recognition

For abnormal gait recognition, pattern-specific features indicating abnormalities are interleaved with the subject-specific differences representing biometric traits. Deep representations are, therefore, prone to overfitting, and the models derived cannot generalize well to new subjects. Furthermore, there is limited availability of abnormal gait data obtained from precise Motion Capture (Mocap) systems because of regulatory issues and slow adaptation of new technologies in health care. On the other hand, data captured from markerless vision sensors or wearable sensors can be obtained in home environments, but noises from such devices may prevent the effective extraction of relevant features. To address these challenges, we propose a cascade of deep architectures that can encode cross-modal and cross-subject transfer for abnormal gait recognition. Cross-modal transfer maps noisy data obtained from RGBD and wearable sensors to accurate 4-D representations of the lower limb and joints obtained from the Mocap system. Subsequently, cross-subject transfer allows disentangling subject-specific from abnormal pattern-specific gait features based on a multiencoder autoencoder architecture. To validate the proposed methodology, we obtained multimodal gait data based on a multicamera motion capture system along with synchronized recordings of electromyography (EMG) data and 4-D skeleton data extracted from a single RGBD camera. Classification accuracy was improved significantly in both Mocap and noisy modalities.

Horses Could Perceive Riding Differently Depending on the Way They Express Poor Welfare in the Stable

This study investigated the relationships between four behavioral and postural indicators of a compromised welfare state in loose boxes (stereotypies, aggressive behaviors toward humans, withdrawn posture reflecting unresponsiveness to the environment, and alert posture indicating hypervigilance) and the way horses perceived riding. This perception was inferred using a survey completed by the usual riding instructor and during a standardized riding session (assessment of behaviors and postures, qualitative behavior assessment (QBA) and characterization of the horses' locomotion using an inertial measurement unit). In accordance with ear and tail positions and the QBA, stereotypic and the most hypervigilant horses in loose boxes seemed to experience a more negative affective state during the riding session compared with nonstereotypic and less hypervigilant animals (P < .02 in all cases). Horses which were aggressive toward humans in loose boxes had higher scores regarding the occurrence of discomfort and defensive behaviors on the survey than nonaggressive horses (P = .03). They also presented higher dorsoventral accelerations at a canter during the riding session (P = .03), requiring the rider to increase his spinal movement (P = .005). These results suggest that aggressive horses may be harder to ride than nonaggressive animals. The expression of unresponsiveness to the environment in loose boxes was related to more reluctance to move forward, as assessed in the survey (P = .006). This study suggests that a compromised welfare state in the stable is related to horses having a more negative perception of riding. This perception could vary depending on the expression of poor welfare.