Gait Recognition via Deep Learning of the Center-of-Pressure Trajectory

Feature Learning Networks for Floor Sensor-based Gait Recognition

Deep learning (DL) has become a powerful tool in many image classification applications but often requires large training sets to achieve high accuracy. For applications where the available data are limited, this can become a severely limiting factor in model performance. To address this limitation, feature learning network approaches that integrate traditional feature extraction methods with DL frameworks have been proposed. In this study, the performances of traditional methods: discrete wavelet transform (DWT), discrete cosine transform (DCT), independent component analysis (ICA), and principal component analysis (PCA); and their corresponding feature networks based on a convolutional neural network (CNN) framework: ScatNet (wavelet scattering network), DCTNet, ICANet, and PCANet, were investigated for use in pressure-based footstep recognition when the limited sample size is available for person authentication. The results show that the feature learning networks (90.6% accuracy) achieved significantly better performance on average than the conventional feature extraction methods (79.7% accuracy) (p < 0.05). Among the different feature networks, PCANet provided the best verification performance, with an accuracy of 92.2%. Feature learning networks are simple and effective approaches that can be a promising solution for applications like floor-based gait recognition in a security access scenario (such as workspace environment and border control) when small amounts of data are available for training models to differentiate between a larger group of users.

Deep Metric Learning for Scalable Gait-Based Person Re-Identification Using Force Platform Data

Walking gait data acquired with force platforms may be used for person re-identification (re-ID) in various authentication, surveillance, and forensics applications. Current force platform-based re-ID systems classify a fixed set of identities (IDs), which presents a problem when IDs are added or removed from the database. We formulated force platform-based re-ID as a deep metric learning (DML) task, whereby a deep neural network learns a feature representation that can be compared between inputs using a distance metric. The force platform dataset used in this study is one of the largest and the most comprehensive of its kind, containing 193 IDs with significant variations in clothing, footwear, walking speed, and time between trials. Several DML model architectures were evaluated in a challenging setting where none of the IDs were seen during training (i.e., zero-shot re-ID) and there was only one prior sample per ID to compare with each query sample. The best architecture was 85% accurate in this setting, though an analysis of changes in walking speed and footwear between measurement instances revealed that accuracy was 28% higher on same-speed, same-footwear comparisons, compared to cross-speed, cross-footwear comparisons. These results demonstrate the potential of DML algorithms for zero-shot re-ID using force platform data, and highlight challenging cases.

User Authentication by Gait Data from Smartphone Sensors Using Hybrid Deep Learning Network

User authentication and verification by gait data based on smartphones’ inertial sensors has gradually attracted increasing attention due to their compact size, portability and affordability. However, the existing approaches often require users to walk on a specific road at a normal walking speed to improve recognition accuracy. In order to recognize gaits under unconstrained conditions on where and how users walk, we proposed a Hybrid Deep Learning Network (HDLN), which combined the advantages of a long short-term memory (LSTM) network and a convolutional neural network (CNN) to reliably extract discriminative features from complex smartphone inertial data. The convergence layer of HDLN was optimized through a spatial pyramid pooling and attention mechanism. The former ensured that the gait features were extracted from more dimensions, and the latter ensured that only important gait information was processed while ignoring unimportant data. Furthermore, we developed an APP that can achieve real-time gait recognition. The experimental results showed that HDLN achieved better performance improvements than CNN, LSTM, DeepConvLSTM and CNN+LSTM by 1.9%, 2.8%, 2.0% and 1.3%, respectively. Furthermore, the experimental results indicated our model’s high scalability and strong suitability in real application scenes.

Automatic Classification of Barefoot and Shod Populations Based on the Foot Metrics and Plantar Pressure Patterns

The human being’s locomotion under the barefoot condition enables normal foot function and lower limb biomechanical performance from a biological evolution perspective. No study has demonstrated the specific differences between habitually barefoot and shod cohorts based on foot morphology and dynamic plantar pressure during walking and running. The present study aimed to assess and classify foot metrics and dynamic plantar pressure patterns of barefoot and shod people via machine learning algorithms. One hundred and forty-six age-matched barefoot (n = 78) and shod (n = 68) participants were recruited for this study. Gaussian Naïve Bayes were selected to identify foot morphology differences between unshod and shod cohorts. The support vector machine (SVM) classifiers based on the principal component analysis (PCA) feature extraction and recursive feature elimination (RFE) feature selection methods were utilized to separate and classify the barefoot and shod populations via walking and running plantar pressure parameters. Peak pressure in the M1-M5 regions during running was significantly higher for the shod participants, increasing 34.8, 37.3, 29.2, 31.7, and 40.1%, respectively. The test accuracy of the Gaussian Naïve Bayes model achieved an accuracy of 93%. The mean 10-fold cross-validation scores were 0.98 and 0.96 for the RFE- and PCA-based SVM models, and both feature extract-based and feature select-based SVM models achieved an accuracy of 95%. The foot shape, especially the forefoot region, was shown to be a valuable classifier of shod and unshod groups. Dynamic pressure patterns during running contribute most to the identification of the two cohorts, especially the forefoot region.

The application of artificial intelligence and custom algorithms with inertial wearable devices for gait analysis and detection of gait-altering pathologies in adults: A scoping review of literature

Background

The purpose of this scoping review was to explore the current applications of objective gait analysis using inertial measurement units, custom algorithms and artificial intelligence algorithms in detecting neurological and musculoskeletal gait altering pathologies from healthy gait patterns.

Methods

Literature searches were conducted of four electronic databases (Medline, PubMed, Embase and Web of Science) to identify studies that assessed the accuracy of these custom gait analysis models with inputs derived from wearable devices. Data was collected according to the preferred reporting items for systematic reviews and meta-analysis statement guidelines.

Results

A total of 23 eligible studies were identified for inclusion in the present review, including 10 custom algorithms articles and 13 artificial intelligence algorithms articles. Nine studies evaluated patients with Parkinson’s disease of varying severity and subtypes. Support vector machine was the commonest adopted artificial intelligence algorithm model, followed by random forest and neural networks. Overall classification accuracy was promising for articles that use artificial intelligence algorithms, with nine articles achieving more than 90% accuracy.

Conclusions

Current applications of artificial intelligence algorithms are reasonably effective discrimination between pathological and non-pathological gait. Of these, machine learning algorithms demonstrate the additional capacity to handle complicated data input, when compared to other custom algorithms. Notably, there has been increasing application of machine learning algorithms for conducting gait analysis. More studies are needed with unsupervised methods and in non-clinical settings to better reflect the community and home-based usage.

Analysis of Center of Pressure Signals by Using Decision Tree and Empirical Mode Decomposition to Predict Falls among Older Adults

Falls put older adults at great risk and are related to the body's sense of balance. This study investigated how to detect the possibility of high fall risk subjects among older adults. The original signal is based on center of pressure (COP) measured using a force plate. The falling group includes 29 subjects who had a history of falls in the year preceding this study or had received high scores on the Short Falls Efficacy Scale (FES). The nonfalling group includes 47 enrollees with no history of falls and who had received low scores on the Short FES. The COP in both the anterior–posterior and mediolateral direction were calculated and analyzed through empirical mode decomposition (EMD) up to six levels. The following five features were extracted and imported to a decision tree algorithm: root-mean-square deviation, median frequency, total frequency power, approximate entropy, and sample entropy. The results showed that there were a larger number of statistically different feature parameters, and a higher classification of accuracy was obtained. With the aid of empirical mode decomposition, the average classification accuracy increased 10% and achieved a level of 99.74% in the training group and 96.77% in the testing group, respectively.

Unsupervised Generative Adversarial Network for Plantar Pressure Image-to-Image Translation

Analyzing human gait from plantar pressure is critical for human health. The majority of works focus on classifying the healthy plantar pattern from unhealthy ones. Different from previous works, we adopt a generative adversarial network to produce healthy plantar pressure image for individual patients. In this work, we do not have pairs of images for training thus we cast the problem as an unsupervised generative adversarial learning task. Our network benefits from multiple components: an encoder-decoder generator, a convolution-based discriminator, a convolution-based evaluation network, and a new term in the loss function to preserve the person's gait style. Our method achieves high performance (99.8%) on the CAD WALK databases which have patients with hallux valgus disease.

Design and Implementation of Deep Learning Based Contactless Authentication System Using Hand Gestures

Hand gestures based sign language digits have several contactless applications. Applications include communication for impaired people, such as elderly and disabled people, health-care applications, automotive user interfaces, and security and surveillance. This work presents the design and implementation of a complete end-to-end deep learning based edge computing system that can verify a user contactlessly using ‘authentication code’. The ‘authentication code’ is an ‘n’ digit numeric code and the digits are hand gestures of sign language digits. We propose a memory-efficient deep learning model to classify the hand gestures of the sign language digits. The proposed deep learning model is based on the bottleneck module which is inspired by the deep residual networks. The model achieves classification accuracy of 99.1% on the publicly available sign language digits dataset. The model is deployed on a Raspberry pi 4 Model B edge computing system to serve as an edge device for user verification. The edge computing system consists of two steps, it first takes input from the camera attached to it in real-time and stores it in the buffer. In the second step, the model classifies the digit with the inference rate of 280 ms, by taking the first image in the buffer as input.

Analysis and best parameters selection for person recognition based on gait model using CNN algorithm and image augmentation

Person Recognition based on Gait Model (PRGM) and motion features is are indeed a challenging and novel task due to their usages and to the critical issues of human pose variation, human body occlusion, camera view variation, etc. In this project, a deep convolution neural network (CNN) was modified and adapted for person recognition with Image Augmentation (IA) technique depending on gait features. Adaptation aims to get best values for CNN parameters to get best CNN model. In Addition to the CNN parameters Adaptation, the design of CNN model itself was adapted to get best model structure; Adaptation in the design was affected the type, the number of layers in CNN and normalization between them. After choosing best parameters and best design, Image augmentation was used to increase the size of train dataset with many copies of the image to boost the number of different images that will be used to train Deep learning algorithms. The tests were achieved using known dataset (Market dataset). The dataset contains sequential pictures of people in different gait status. The image in CNN model as matrix is extracted to many images or matrices by the convolution, so dataset size may be bigger by hundred times to make the problem a big data issue. In this project, results show that adaptation has improved the accuracy of person recognition using gait model comparing to model without adaptation. In addition, dataset contains images of person carrying things. IA technique improved the model to be robust to some variations such as image dimensions (quality and resolution), rotations and carried things by persons. Results for 200 persons recognition, validation accuracy was about 82% without IA and 96.23 with IA. For 800 persons recognition, validation accuracy was 93.62% without IA.

Working Principle and Performance of a Scalable Gravimetric System for the Monitoring of Access to Public Places

Here, we propose a novel application of a low-cost robust gravimetric system for public place access monitoring purposes. The proposed solution is intended to be exploited in a multi-sensor scenario, where heterogeneous information, coming from different sources (e.g., metal detectors and surveillance cameras), are collected in a central data fusion unit to obtain a more detailed and accurate evaluation of notable events. Specifically, the word “notable” refers essentially to two event categories: the first category is represented by irregular events, corresponding typically to multiple people passing together through a security gate; the second category includes some event subsets, whose notification can be interesting for assistance provision (in the case of people with disabilities), or for statistical analysis. The employed gravimetric sensor, compared to other devices existing in the literature, exhibits a simple scalable robust structure, made up of an array of rigid steel plates, each laid on four load cells. We developed a tailored hardware and software to individually acquire the load cell signals, and to post-process the data to formulate a classification of the notable events. The results are encouraging, showing a remarkable detectability of irregularities (95.3% of all the test cases) and a satisfactory identification of the other event types.

Human Attribute Recognition— A Comprehensive Survey

Human Attribute Recognition (HAR) is a highly active research field in computer vision and pattern recognition domains with various applications such as surveillance or fashion. Several approaches have been proposed to tackle the particular challenges in HAR. However, these approaches have dramatically changed over the last decade, mainly due to the improvements brought by deep learning solutions. To provide insights for future algorithm design and dataset collections, in this survey, (1) we provide an in-depth analysis of existing HAR techniques, concerning the advances proposed to address the HAR’s main challenges; (2) we provide a comprehensive discussion over the publicly available datasets for the development and evaluation of novel HAR approaches; (3) we outline the applications and typical evaluation metrics used in the HAR context.