Multi-View Gait Image Generation for Cross-View Gait Recognition

Interpretable machine learning comprehensive human gait deterioration analysis

Introduction

Gait analysis, an expanding research area, employs non-invasive sensors and machine learning techniques for a range of applications. In this study, we investigate the impact of cognitive decline conditions on gait performance, drawing connections between gait deterioration in Parkinson's Disease (PD) and healthy individuals dual tasking.

Methods

We employ Explainable Artificial Intelligence (XAI) specifically Layer-Wise Relevance Propagation (LRP), in conjunction with Convolutional Neural Networks (CNN) to interpret the intricate patterns in gait dynamics influenced by cognitive loads.

Results

We achieved classification accuracies of 98% F1 scores for PD dataset and 95.5% F1 scores for the combined PD dataset. Furthermore, we explore the significance of cognitive load in healthy gait analysis, resulting in robust classification accuracies of 90% ± 10% F1 scores for subject cognitive load verification. Our findings reveal significant alterations in gait parameters under cognitive decline conditions, highlighting the distinctive patterns associated with PD-related gait impairment and those induced by multitasking in healthy subjects. Through advanced XAI techniques (LRP), we decipher the underlying features contributing to gait changes, providing insights into specific aspects affected by cognitive decline.

Discussion

Our study establishes a novel perspective on gait analysis, demonstrating the applicability of XAI in elucidating the shared characteristics of gait disturbances in PD and dual-task scenarios in healthy individuals. The interpretability offered by XAI enhances our ability to discern subtle variations in gait patterns, contributing to a more nuanced comprehension of the factors influencing gait dynamics in PD and dual-task conditions, emphasizing the role of XAI in unraveling the intricacies of gait control.

Regional Time-Series Coding Network and Multi-View Image Generation Network for Short-Time Gait Recognition

Gait recognition is one of the important research directions of biometric authentication technology. However, in practical applications, the original gait data is often short, and a long and complete gait video is required for successful recognition. Also, the gait images from different views have a great influence on the recognition effect. To address the above problems, we designed a gait data generation network for expanding the cross-view image data required for gait recognition, which provides sufficient data input for feature extraction branching with gait silhouette as the criterion. In addition, we propose a gait motion feature extraction network based on regional time-series coding. By independently time-series coding the joint motion data within different regions of the body, and then combining the time-series data features of each region with secondary coding, we obtain the unique motion relationships between regions of the body. Finally, bilinear matrix decomposition pooling is used to fuse spatial silhouette features and motion time-series features to obtain complete gait recognition under shorter time-length video input. We use the OUMVLP-Pose and CASIA-B datasets to validate the silhouette image branching and motion time-series branching, respectively, and employ evaluation metrics such as IS entropy value and Rank-1 accuracy to demonstrate the effectiveness of our design network. Finally, we also collect gait-motion data in the real world and test them in a complete two-branch fusion network. The experimental results show that the network we designed can effectively extract the time-series features of human motion and achieve the expansion of multi-view gait data. The real-world tests also prove that our designed method has good results and feasibility in the problem of gait recognition with short-time video as input data.

Multi gait recognition using Clustering based Faster Regions-Convolutional Neural Network

Gait recognition is the process of recognizing a person based on their walking style. Each person’s walking gait is distinctive and cannot be imitated by others. However, the walking motion of a person will be changed based on their behaviour but their walking pattern doesn’t change. In this paper, a novel Clustering based Faster RCNN has been proposed to identify the single, double and multi-gait. The gait images from the publicly available dataset are pre-processed using Multi scale Retinex (MSR) to reduce the noise artifacts. The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN. The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features. Fuzzy K-means clustering is used to group the features based on their labels, and it specifies the features acquired using CNN and RPN as input. Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking pattern. The proposed Clustering based Faster RCNN net achieves the high accuracy rate of 98.74% and 99.19% for suspicious and non-suspicious walking pattern respectively. The proposed Clustering based Faster RCNN model was compared with other traditional models like CNN, U-net, Fab net and Fast R-CNN. The proposed Clustering based Faster RCNN model improves the overall accuracy of 8.86% 33.77% 3.12% and 5.48% better than mmGait, LSTM Net, STDNN and RNN respectively.

Deep Gait Recognition: A Survey

Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

Advances in Vision-Based Gait Recognition: From Handcrafted to Deep Learning

Identifying people’s identity by using behavioral biometrics has attracted many researchers’ attention in the biometrics industry. Gait is a behavioral trait, whereby an individual is identified based on their walking style. Over the years, gait recognition has been performed by using handcrafted approaches. However, due to several covariates’ effects, the competence of the approach has been compromised. Deep learning is an emerging algorithm in the biometrics field, which has the capability to tackle the covariates and produce highly accurate results. In this paper, a comprehensive overview of the existing deep learning-based gait recognition approach is presented. In addition, a summary of the performance of the approach on different gait datasets is provided.

Human Gait Analysis: A Sequential Framework of Lightweight Deep Learning and Improved Moth-Flame Optimization Algorithm

Human gait recognition has emerged as a branch of biometric identification in the last decade, focusing on individuals based on several characteristics such as movement, time, and clothing. It is also great for video surveillance applications. The main issue with these techniques is the loss of accuracy and time caused by traditional feature extraction and classification. With advances in deep learning for a variety of applications, particularly video surveillance and biometrics, we proposed a lightweight deep learning method for human gait recognition in this work. The proposed method includes sequential steps-pretrained deep models selection of features classification. Two lightweight pretrained models are initially considered and fine-tuned in terms of additional layers and freezing some middle layers. Following that, models were trained using deep transfer learning, and features were engineered on fully connected and average pooling layers. The fusion is performed using discriminant correlation analysis, which is then optimized using an improved moth-flame optimization algorithm. For final classification, the final optimum features are classified using an extreme learning machine (ELM). The experiments were carried out on two publicly available datasets, CASIA B and TUM GAID, and yielded an average accuracy of 91.20 and 98.60%, respectively. When compared to recent state-of-the-art techniques, the proposed method is found to be more accurate.

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

In recent decades, although the research on gait recognition of lower limb exoskeleton robot has been widely developed, there are still limitations in rehabilitation training and clinical practice. The emergence of interactive information fusion technology provides a new research idea for the solution of this problem, and it is also the development trend in the future. In order to better explore the issue, this paper summarizes gait recognition based on interactive information fusion of lower limb exoskeleton robots. This review introduces the current research status, methods, and directions for information acquisition, interaction, fusion, and gait recognition of exoskeleton robots. The content involves the research progress of information acquisition methods, sensor placements, target groups, lower limb sports biomechanics, interactive information fusion, and gait recognition model. Finally, the current challenges, possible solutions, and promising prospects are analysed and discussed, which provides a useful reference resource for the study of interactive information fusion and gait recognition of rehabilitation exoskeleton robots.