Advances in Vision-Based Gait Recognition: From Handcrafted to Deep Learning

MFCF-Gait: Small Silhouette-Sensitive Gait Recognition Algorithm Based on Multi-Scale Feature Cross-Fusion

Gait recognition based on gait silhouette profiles is currently a major approach in the field of gait recognition. In previous studies, models typically used gait silhouette images sized at 64 × 64 pixels as input data. However, in practical applications, cases may arise where silhouette images are smaller than 64 × 64, leading to a loss in detail information and significantly affecting model accuracy. To address these challenges, we propose a gait recognition system named Multi-scale Feature Cross-Fusion Gait (MFCF-Gait). At the input stage of the model, we employ super-resolution algorithms to preprocess the data. During this process, we observed that different super-resolution algorithms applied to larger silhouette images also affect training outcomes. Improved super-resolution algorithms contribute to enhancing model performance. In terms of model architecture, we introduce a multi-scale feature cross-fusion network model. By integrating low-level feature information from higher-resolution images with high-level feature information from lower-resolution images, the model emphasizes smaller-scale details, thereby improving recognition accuracy for smaller silhouette images. The experimental results on the CASIA-B dataset demonstrate significant improvements. On 64 × 64 silhouette images, the accuracies for NM, BG, and CL states reached 96.49%, 91.42%, and 78.24%, respectively. On 32 × 32 silhouette images, the accuracies were 94.23%, 87.68%, and 71.57%, respectively, showing notable enhancements.

A hybrid human recognition framework using machine learning and deep neural networks

Faces are a crucial environmental trigger. They communicate information about several key features, including identity. However, the 2019 coronavirus pandemic (COVID-19) significantly affected how we process faces. To prevent viral spread, many governments ordered citizens to wear masks in public. In this research, we focus on identifying individuals from images or videos by comparing facial features, identifying a person's biometrics, and reducing the weaknesses of person recognition technology, for example when a person does not look directly at the camera, the lighting is poor, or the person has effectively covered their face. Consequently, we propose a hybrid approach of detecting either a person with or without a mask, a person who covers large parts of their face, and a person based on their gait via deep and machine learning algorithms. The experimental results are excellent compared to the current face and gait detectors. We achieved success of between 97% and 100% in the detection of face and gait based on F1 score, precision, and recall. Compared to the baseline CNN system, our approach achieves extremely high recognition accuracy.

Gait-ViT: Gait Recognition with Vision Transformer

Identifying an individual based on their physical/behavioral characteristics is known as biometric recognition. Gait is one of the most reliable biometrics due to its advantages, such as being perceivable at a long distance and difficult to replicate. The existing works mostly leverage Convolutional Neural Networks for gait recognition. The Convolutional Neural Networks perform well in image recognition tasks; however, they lack the attention mechanism to emphasize more on the significant regions of the image. The attention mechanism encodes information in the image patches, which facilitates the model to learn the substantial features in the specific regions. In light of this, this work employs the Vision Transformer (ViT) with an attention mechanism for gait recognition, referred to as Gait-ViT. In the proposed Gait-ViT, the gait energy image is first obtained by averaging the series of images over the gait cycle. The images are then split into patches and transformed into sequences by flattening and patch embedding. Position embedding, along with patch embedding, are applied on the sequence of patches to restore the positional information of the patches. Subsequently, the sequence of vectors is fed to the Transformer encoder to produce the final gait representation. As for the classification, the first element of the sequence is sent to the multi-layer perceptron to predict the class label. The proposed method obtained 99.93% on CASIA-B, 100% on OU-ISIR D and 99.51% on OU-LP, which exhibit the ability of the Vision Transformer model to outperform the state-of-the-art methods.

Detecting Gait Events from Accelerations Using Reservoir Computing

Segmenting the gait cycle into multiple phases using gait event detection (GED) is a well-researched subject with many accurate algorithms. However, the algorithms that are able to perform accurate and robust GED for real-life environments and physical diseases tend to be too complex for their implementation on simple hardware systems limited in computing power and memory, such as those used in wearable devices. This study focuses on a numerical implementation of a reservoir computing (RC) algorithm called the echo state network (ESN) that is based on simple computational steps that are easy to implement on portable hardware systems for real-time detection. RC is a neural network method that is widely used for signal processing applications and uses a fast-training method based on a ridge regression adapted to the large quantity and variety of IMU data needed to use RC in various real-life environment GED. In this study, an ESN was used to perform offline GED with gait data from IMU and ground force sensors retrieved from three databases for a total of 28 healthy adults and 15 walking conditions. Our main finding is that despite its low complexity, ESN is robust for GED, with performance comparable to other state-of-the-art algorithms. Our results show the ESN is robust enough to obtain good detection results in all conditions if the algorithm is trained with variable data that match those conditions. The distribution of the mean absolute errors (MAE) between the detection times from the ESN and the force sensors were between 40 and 120 ms for 6 defined gait events (95th percentile). We compared our ESN with four different state-of-the-art algorithms from the literature. The ESN obtained a MAE not more than 10 ms above three other reference algorithms for normal walking indoor and outdoor conditions and yielded the 2nd lowest MAE and the 2nd highest true positive rate and specificity when applied to outdoor walking and running conditions. Our work opens the door to using the ESN as a GED for applications in wearable sensors for long-term patient monitoring.