Gait Recognition and Understanding Based on Hierarchical Temporal Memory Using 3D Gait Semantic Folding

Gender Recognition Based on Gradual and Ensemble Learning from Multi-View Gait Energy Images and Poses

Image-based gender classification is very useful in many applications, such as intelligent surveillance, micromarketing, etc. One common approach is to adopt a machine learning algorithm to recognize the gender class of the captured subject based on spatio-temporal gait features extracted from the image. The image input can be generated from the video of the walking cycle, e.g., gait energy image (GEI). Recognition accuracy depends on the similarity of intra-class GEIs, as well as the dissimilarity of inter-class GEIs. However, we observe that, at some viewing angles, the GEIs of both gender classes are very similar. Moreover, the GEI does not exhibit a clear appearance of posture. We postulate that distinctive postures of the walking cycle can provide additional and valuable information for gender classification. This paper proposes a gender classification framework that exploits multiple inputs of the GEI and the characteristic poses of the walking cycle. The proposed framework is a cascade network that is capable of gradually learning the gait features from images acquired in multiple views. The cascade network contains a feature extractor and gender classifier. The multi-stream feature extractor network is trained to extract features from the multiple input images. Features are then fed to the classifier network, which is trained with ensemble learning. We evaluate and compare the performance of our proposed framework with state-of-the-art gait-based gender classification methods on benchmark datasets. The proposed framework outperforms other methods that only utilize a single input of the GEI or pose.

IoT and Deep Learning-Based Farmer Safety System

Farming is a fundamental factor driving economic development in most regions of the world. As in agricultural activity, labor has always been hazardous and can result in injury or even death. This perception encourages farmers to use proper tools, receive training, and work in a safe environment. With the wearable device as an Internet of Things (IoT) subsystem, the device can read sensor data as well as compute and send information. We investigated the validation and simulation dataset to determine whether accidents occurred with farmers by applying the Hierarchical Temporal Memory (HTM) classifier with each dataset input from the quaternion feature that represents 3D rotation. The performance metrics analysis showed a significant 88.00% accuracy, precision of 0.99, recall of 0.04, F_Score of 0.09, average Mean Square Error (MSE) of 5.10, Mean Absolute Error (MAE) of 0.19, and a Root Mean Squared Error (RMSE) of 1.51 for the validation dataset, 54.00% accuracy, precision of 0.97, recall of 0.50, F_Score of 0.66, MSE = 0.06, MAE = 3.24, and = 1.51 for the Farming-Pack motion capture (mocap) dataset. The computational framework with wearable device technology connected to ubiquitous systems, as well as statistical results, demonstrate that our proposed method is feasible and effective in solving the problem's constraints in a time series dataset that is acceptable and usable in a real rural farming environment for optimal solutions.

User Authentication by Gait Data from Smartphone Sensors Using Hybrid Deep Learning Network

User authentication and verification by gait data based on smartphones’ inertial sensors has gradually attracted increasing attention due to their compact size, portability and affordability. However, the existing approaches often require users to walk on a specific road at a normal walking speed to improve recognition accuracy. In order to recognize gaits under unconstrained conditions on where and how users walk, we proposed a Hybrid Deep Learning Network (HDLN), which combined the advantages of a long short-term memory (LSTM) network and a convolutional neural network (CNN) to reliably extract discriminative features from complex smartphone inertial data. The convergence layer of HDLN was optimized through a spatial pyramid pooling and attention mechanism. The former ensured that the gait features were extracted from more dimensions, and the latter ensured that only important gait information was processed while ignoring unimportant data. Furthermore, we developed an APP that can achieve real-time gait recognition. The experimental results showed that HDLN achieved better performance improvements than CNN, LSTM, DeepConvLSTM and CNN+LSTM by 1.9%, 2.8%, 2.0% and 1.3%, respectively. Furthermore, the experimental results indicated our model’s high scalability and strong suitability in real application scenes.

Application of Machine Vision in Classifying Gait Frailty Among Older Adults

Background: Frail older adults have an increased risk of adverse health outcomes and premature death. They also exhibit altered gait characteristics in comparison with healthy individuals. Methods: In this study, we created a Fried's frailty phenotype (FFP) labelled casual walking video set of older adults based on the West China Health and Aging Trend study. A series of hyperparameters in machine vision models were evaluated for body key point extraction (AlphaPose), silhouette segmentation (Pose2Seg, DPose2Seg, and Mask R-CNN), gait feature extraction (Gaitset, LGaitset, and DGaitset), and feature classification (AlexNet and VGG16), and were highly optimised during analysis of gait sequences of the current dataset. Results: The area under the curve (AUC) of the receiver operating characteristic (ROC) at the physical frailty state identification task for AlexNet was 0.851 (0.827-0.8747) and 0.901 (0.878-0.920) in macro and micro, respectively, and was 0.855 (0.834-0.877) and 0.905 (0.886-0.925) for VGG16 in macro and micro, respectively. Furthermore, this study presents the machine vision method equipped with better predictive performance globally than age and grip strength, as well as than 4-m-walking-time in healthy and pre-frailty classifying. Conclusion: The gait analysis method in this article is unreported and provides promising original tool for frailty and pre-frailty screening with the characteristics of convenience, objectivity, rapidity, and non-contact. These methods can be extended to any gait-related disease identification processes, as well as in-home health monitoring.