A study on gait-based gender classification

A Multi-Activity Fusion Approach for Gender Recognition based on Human Activity

Due to its advantages in numerous industries, including healthcare, sports, rehabilitation, and wearable electronics, gender recognition has garnered a lot of attention in the last ten years. The gender recognition method described in this study uses a wearable sensor device with inertial measurement units to record a variety of activities. The system consists of five sensors that are mounted to the upper and lower bodies while performing seven standing, walking, and climbing exercises that are meant to replicate daily activity. To create a model for gender recognition, we carried out an extensive study based on supervised machine learning. This study identifies a collection of sensor locations and behaviours to better precisely classify gender. Gender classification based on single activity was performed using Random Forest Classifier (RFC) and Support Vector Machines (SVM). Maximum accuracy of 92.06% was gained using Random Forest Classifier for the sensor located at the ankle when walking. Multi-activity based gender classification outperformed former by achieving an accuracy of 94.13% using RFC. This was for the activity combination of Romberg test eyes open, Single leg stance eyes open and Staircase up and down.

Person Recognition Based on Deep Gait: A Survey

Gait recognition, also known as walking pattern recognition, has expressed deep interest in the computer vision and biometrics community due to its potential to identify individuals from a distance. It has attracted increasing attention due to its potential applications and non-invasive nature. Since 2014, deep learning approaches have shown promising results in gait recognition by automatically extracting features. However, recognizing gait accurately is challenging due to the covariate factors, complexity and variability of environments, and human body representations. This paper provides a comprehensive overview of the advancements made in this field along with the challenges and limitations associated with deep learning methods. For that, it initially examines the various gait datasets used in the literature review and analyzes the performance of state-of-the-art techniques. After that, a taxonomy of deep learning methods is presented to characterize and organize the research landscape in this field. Furthermore, the taxonomy highlights the basic limitations of deep learning methods in the context of gait recognition. The paper is concluded by focusing on the present challenges and suggesting several research directions to improve the performance of gait recognition in the future.

Identification of the Visually Prominent Gait Parameters for Forensic Gait Analysis

Walking patterns can be used as a key parameter in identifying individuals, as it varies visually depending on one’s body size as well as their habits, gender, and age group. In this study, we measure the gait characteristics of a large number of subjects using 34 visual parameters to identify significant parameters that can be used to distinguish individual walking features. We recorded 291 subjects’ walking on a constructed footpath using four video cameras, and data on parameters was calculated at the points of double support, toe-off, and heel-strike. K-means Clustering Analysis and ANOVA were conducted to determine the difference between age, gender, and BMI. As a result, we confirm that parameters related to the spine, neck, and feet are useful for identifying individuals. In the comparative analysis between age groups, the older the age, the more significant variables appeared in the upper body. The difference between genders showed significant parameters in both the upper and lower bodies of males. Similarly, among the large BMI groups, we also derived significant results in the upper and lower bodies. The key parameters derived from this study can be used more effectively in the real-world visual analysis of gait, as the walking characteristics of a large number of subjects have been measured with a similar view as real-world CCTV. This study will be effectively utilized as a foundation for future research attempting to identify people through their gait by distinguishing major gait characteristic differences.

Decoding Individual differences and musical preference via music-induced movement

Movement is a universal response to music, with dance often taking place in social settings. Although previous work has suggested that socially relevant information, such as personality and gender, are encoded in dance movement, the generalizability of previous work is limited. The current study aims to decode dancers’ gender, personality traits, and music preference from music-induced movements. We propose a method that predicts such individual difference from free dance movements, and demonstrate the robustness of the proposed method by using two data sets collected using different musical stimuli. In addition, we introduce a novel measure to explore the relative importance of different joints in predicting individual differences. Results demonstrated near perfect classification of gender, and notably high prediction of personality and music preferences. Furthermore, learned models demonstrated generalizability across datasets highlighting the importance of certain joints in intrinsic movement patterns specific to individual differences. Results further support theories of embodied music cognition and the role of bodily movement in musical experiences by demonstrating the influence of gender, personality, and music preferences on embodied responses to heard music.

Deep Learning Based Real Age and Gender Estimation from Unconstrained Face Image towards Smart Store Customer Relationship Management

The COVID-19 pandemic markedly changed the human shopping nature, necessitating a contactless shopping system to curb the spread of the contagious disease efficiently. Consequently, a customer opts for a store where it is possible to avoid physical contacts and shorten the shopping process with extended services such as personalized product recommendations. Automatic age and gender estimation of a customer in a smart store strongly benefit the consumer by providing personalized advertisement and product recommendation; similarly, it aids the smart store proprietor to promote sales and develop an inventory perpetually for the future retail. In our paper, we propose a deep learning-founded enterprise solution for smart store customer relationship management (CRM), which allows us to predict the age and gender from a customer’s face image taken in an unconstrained environment to facilitate the smart store’s extended services, as it is expected for a modern venture. For the age estimation problem, we mitigate the data sparsity problem of the large public IMDB-WIKI dataset by image enhancement from another dataset and perform data augmentation as required. We handle our classification tasks utilizing an empirically leading pre-trained convolutional neural network (CNN), the VGG-16 network, and incorporate batch normalization. Especially, the age estimation task is posed as a deep classification problem followed by a multinomial logistic regression first-moment refinement. We validate our system for two standard benchmarks, one for each task, and demonstrate state-of-the-art performance for both real age and gender estimation.

Human Gender Classification Using Transfer Learning via Pareto Frontier CNN Networks

Human gender is deemed as a prime demographic trait due to its various usage in the practical domain. Human gender classification in an unconstrained environment is a sophisticated task due to large variations in the image scenarios. Due to the multifariousness of internet images, the classification accuracy suffers from traditional machine learning methods. The aim of this research is to streamline the gender classification process using the transfer learning concept. This research proposes a framework that performs automatic gender classification in unconstrained internet images deploying Pareto frontier deep learning networks; GoogleNet, SqueezeNet, and ResNet50. We analyze the experiment with three different Pareto frontier Convolutional Neural Network (CNN) models pre-trained on ImageNet. The massive experiments demonstrate that the performance of the Pareto frontier CNN networks is remarkable in the unconstrained internet image dataset as well as in the frontal images that pave the way to developing an automatic gender classification system.

Droplet-Transmitted Infection Risk Ranking Based on Close Proximity Interaction

We propose an automatic method to identify people who are potentially-infected by droplet-transmitted diseases. This high-risk group of infection was previously identified by conducting large-scale visits/interviews, or manually screening among tons of recorded surveillance videos. Both are time-intensive and most likely to delay the control of communicable diseases like influenza. In this paper, we address this challenge by solving a multi-tasking problem from the captured surveillance videos. This multi-tasking framework aims to model the principle of Close Proximity Interaction and thus infer the infection risk of individuals. The complete workflow includes three essential sub-tasks: (1) person re-identification (REID), to identify the diagnosed patient and infected individuals across different cameras, (2) depth estimation, to provide a spatial knowledge of the captured environment, (3) pose estimation, to evaluate the distance between the diagnosed and potentially-infected subjects. Our method significantly reduces the time and labor costs. We demonstrate the advantages of high accuracy and efficiency of our method. Our method is expected to be effective in accelerating the process of identifying the potentially infected group and ultimately contribute to the well-being of public health.

A Multi-Task Framework for Facial Attributes Classification through End-to-End Face Parsing and Deep Convolutional Neural Networks

Human face image analysis is an active research area within computer vision. In this paper we propose a framework for face image analysis, addressing three challenging problems of race, age, and gender recognition through face parsing. We manually labeled face images for training an end-to-end face parsing model through Deep Convolutional Neural Networks. The deep learning-based segmentation model parses a face image into seven dense classes. We use the probabilistic classification method and created probability maps for each face class. The probability maps are used as feature descriptors. We trained another Convolutional Neural Network model by extracting features from probability maps of the corresponding class for each demographic task (race, age, and gender). We perform extensive experiments on state-of-the-art datasets and obtained much better results as compared to previous results.

From Emotions to Mood Disorders: A Survey on Gait Analysis Methodology

Mood disorders affect more than 300 million people worldwide and can cause devastating consequences. Elderly people and patients with neurological conditions are particularly susceptible to depression. Gait and body movements can be affected by mood disorders, and thus they can be used as a surrogate sign, as well as an objective index for pervasive monitoring of emotion and mood disorders in daily life. Here we review evidence that demonstrates the relationship between gait, emotions and mood disorders, highlighting the potential of a multimodal approach that couples gait data with physiological signals and home-based monitoring for early detection and management of mood disorders. This could enhance self-awareness, enable the development of objective biomarkers that identify high risk subjects and promote subject-specific treatment.

A Unified Framework for Head Pose, Age and Gender Classification through End-to-End Face Segmentation

Accurate face segmentation strongly benefits the human face image analysis problem. In this paper we propose a unified framework for face image analysis through end-to-end semantic face segmentation. The proposed framework contains a set of stack components for face understanding, which includes head pose estimation, age classification, and gender recognition. A manually labeled face data-set is used for training the Conditional Random Fields (CRFs) based segmentation model. A multi-class face segmentation framework developed through CRFs segments a facial image into six parts. The probabilistic classification strategy is used, and probability maps are generated for each class. The probability maps are used as features descriptors and a Random Decision Forest (RDF) classifier is modeled for each task (head pose, age, and gender). We assess the performance of the proposed framework on several data-sets and report better results as compared to the previously reported results.

Automatic Gender Classification through Face Segmentation

Automatic gender classification is challenging due to large variations of face images, particularly in the un-constrained scenarios. In this paper, we propose a framework which first segments a face image into face parts, and then performs automatic gender classification. We trained a Conditional Random Fields (CRFs) based segmentation model through manually labeled face images. The CRFs based model is used to segment a face image into six different classes—mouth, hair, eyes, nose, skin, and back. The probabilistic classification strategy (PCS) is used, and probability maps are created for all six classes. We use the probability maps as gender descriptors and trained a Random Decision Forest (RDF) classifier, which classifies the face images as either male or female. The performance of the proposed framework is assessed on four publicly available datasets, namely Adience, LFW, FERET, and FEI, with results outperforming state-of-the-art (SOA).

Predicting body movements for person identification under different walking conditions

Human motion during walking provides biometric information which can be utilized to quantify the similarity between two persons or identify a person. The purpose of this study was to develop a method for identifying a person using their walking motion when another walking motion under different conditions is given. This type of situation occurs frequently in forensic gait science. Twenty-eight subjects were asked to walk in a gait laboratory, and the positions of their joints were tracked using a three-dimensional motion capture system. The subjects repeated their walking motion both without a weight and with a tote bag weighing a total of 5% of their body weight in their right hand. The positions of 17 anatomical landmarks during two cycles of a gait trial were generated to form a gait vector. We developed two different linear transformation methods to determine the functional relationship between the normal gait vectors and the tote-bag gait vectors from the collected gait data, one using linear transformations and the other using partial least squares regression. These methods were validated by predicting the tote-bag gait vector given a normal gait vector of a person, accomplished by calculating the Euclidean distance between the predicted vector to the measured tote-bag gait vector of the same person. The mean values of the prediction scores for the two methods were 96.4 and 95.0, respectively. This study demonstrated the potential for identifying a person based on their walking motion, even under different walking conditions.

Demographic Analysis from Biometric Data: Achievements, Challenges, and New Frontiers

Biometrics is the technique of automatically recognizing individuals based on their biological or behavioral characteristics. Various biometric traits have been introduced and widely investigated, including fingerprint, iris, face, voice, palmprint, gait and so forth. Apart from identity, biometric data may convey various other personal information, covering affect, age, gender, race, accent, handedness, height, weight, etc. Among these, analysis of demographics (age, gender, and race) has received tremendous attention owing to its wide real-world applications, with significant efforts devoted and great progress achieved. This survey first presents biometric demographic analysis from the standpoint of human perception, then provides a comprehensive overview of state-of-the-art advances in automated estimation from both academia and industry. Despite these advances, a number of challenging issues continue to inhibit its full potential. We second discuss these open problems, and finally provide an outlook into the future of this very active field of research by sharing some promising opportunities.

Gender Recognition from Human-Body Images Using Visible-Light and Thermal Camera Videos Based on a Convolutional Neural Network for Image Feature Extraction

Extracting powerful image features plays an important role in computer vision systems. Many methods have previously been proposed to extract image features for various computer vision applications, such as the scale-invariant feature transform (SIFT), speed-up robust feature (SURF), local binary patterns (LBP), histogram of oriented gradients (HOG), and weighted HOG. Recently, the convolutional neural network (CNN) method for image feature extraction and classification in computer vision has been used in various applications. In this research, we propose a new gender recognition method for recognizing males and females in observation scenes of surveillance systems based on feature extraction from visible-light and thermal camera videos through CNN. Experimental results confirm the superiority of our proposed method over state-of-the-art recognition methods for the gender recognition problem using human body images.

Enhanced Gender Recognition System Using an Improved Histogram of Oriented Gradient (HOG) Feature from Quality Assessment of Visible Light and Thermal Images of the Human Body

With higher demand from users, surveillance systems are currently being designed to provide more information about the observed scene, such as the appearance of objects, types of objects, and other information extracted from detected objects. Although the recognition of gender of an observed human can be easily performed using human perception, it remains a difficult task when using computer vision system images. In this paper, we propose a new human gender recognition method that can be applied to surveillance systems based on quality assessment of human areas in visible light and thermal camera images. Our research is novel in the following two ways: First, we utilize the combination of visible light and thermal images of the human body for a recognition task based on quality assessment. We propose a quality measurement method to assess the quality of image regions so as to remove the effects of background regions in the recognition system. Second, by combining the features extracted using the histogram of oriented gradient (HOG) method and the measured qualities of image regions, we form a new image features, called the weighted HOG (wHOG), which is used for efficient gender recognition. Experimental results show that our method produces more accurate estimation results than the state-of-the-art recognition method that uses human body images.

Fall Down Detection Under Smart Home System

Medical technology makes an inevitable trend for the elderly population, therefore the intelligent home care is an important direction for science and technology development, in particular, elderly in-home safety management issues become more and more important. In this research, a low of operation algorithm and using the triangular pattern rule are proposed, then can quickly detect fall-down movements of humanoid by the installation of a robot with camera vision at home that will be able to judge the fall-down movements of in-home elderly people in real time. In this paper, it will present a preliminary design and experimental results of fall-down movements from body posture that utilizes image pre-processing and three triangular-mass-central points to extract the characteristics. The result shows that the proposed method would adopt some characteristic value and the accuracy can reach up to 90 % for a single character posture. Furthermore the accuracy can be up to 100 % when a continuous-time sampling criterion and support vector machine (SVM) classifier are used.

EAR BIOMETRICS AND SPARSE REPRESENTATION BASED ON SMOOTHED l0 NORM

Ear biometrics attracted the attention of researchers in computer vision and machine learning for its use in many applications. In this paper, we present a fully automated system for recognition from ear images based upon sparse representation. In sparse representation, extracted features from the training data is used to develop a dictionary. Classification is achieved by representing the extracted features of the test data as a linear combination of entries in the dictionary. In fact, there are many solutions for this problem and the goal is to find the sparsest solution. We use a relatively new algorithm named smoothed l0 norm to find the sparsest solution and Gabor wavelet features are used for building the dictionary. Furthermore, we expand the proposed approach for gender classification from ear images. Several researches have addressed this issue based on facial images. We introduce a novel approach based on majority voting for gender classification. Experimental results conducted on the University of Notre Dame (UND) collection J data set, containing large appearance, pose, and lighting variations, resulted in a gender classification rate of 89.49%. Furthermore, the proposed method is evaluated on the WVU data set and classification rates for different view angles are presented. Results show improvement and great robustness in gender classification over existing methods.

Average gait differential image based human recognition

The difference between adjacent frames of human walking contains useful information for human gait identification. Based on the previous idea a silhouettes difference based human gait recognition method named as average gait differential image (AGDI) is proposed in this paper. The AGDI is generated by the accumulation of the silhouettes difference between adjacent frames. The advantage of this method lies in that as a feature image it can preserve both the kinetic and static information of walking. Comparing to gait energy image (GEI), AGDI is more fit to representation the variation of silhouettes during walking. Two-dimensional principal component analysis (2DPCA) is used to extract features from the AGDI. Experiments on CASIA dataset show that AGDI has better identification and verification performance than GEI. Comparing to PCA, 2DPCA is a more efficient and less memory storage consumption feature extraction method in gait based recognition.

Gait correlation analysis based human identification

Human gait identification aims to identify people by a sequence of walking images. Comparing with fingerprint or iris based identification, the most important advantage of gait identification is that it can be done at a distance. In this paper, silhouette correlation analysis based human identification approach is proposed. By background subtracting algorithm, the moving silhouette figure can be extracted from the walking images sequence. Every pixel in the silhouette has three dimensions: horizontal axis (x), vertical axis (y), and temporal axis (t). By moving every pixel in the silhouette image along these three dimensions, we can get a new silhouette. The correlation result between the original silhouette and the new one can be used as the raw feature of human gait. Discrete Fourier transform is used to extract features from this correlation result. Then, these features are normalized to minimize the affection of noise. Primary component analysis method is used to reduce the features' dimensions. Experiment based on CASIA database shows that this method has an encouraging recognition performance.

Canonical Correlation Analysis on Riemannian Manifolds and Its Applications

Canonical correlation analysis (CCA) is a widely used statistical technique to capture correlations between two sets of multi-variate random variables and has found a multitude of applications in computer vision, medical imaging and machine learning. The classical formulation assumes that the data live in a pair of vector spaces which makes its use in certain important scientific domains problematic. For instance, the set of symmetric positive definite matrices (SPD), rotations and probability distributions, all belong to certain curved Riemannian manifolds where vector-space operations are in general not applicable. Analyzing the space of such data via the classical versions of inference models is rather sub-optimal. But perhaps more importantly, since the algorithms do not respect the underlying geometry of the data space, it is hard to provide statistical guarantees (if any) on the results. Using the space of SPD matrices as a concrete example, this paper gives a principled generalization of the well known CCA to the Riemannian setting. Our CCA algorithm operates on the product Riemannian manifold representing SPD matrix-valued fields to identify meaningful statistical relationships on the product Riemannian manifold. As a proof of principle, we present results on an Alzheimer's disease (AD) study where the analysis task involves identifying correlations across diffusion tensor images (DTI) and Cauchy deformation tensor fields derived from T1-weighted magnetic resonance (MR) images.

On the Generalization Power of Face and Gait in Gender Recognition

Human face/gait-based gender recognition has been intensively studied in the previous literatures, yet most of them are based on the same database. Although nearly perfect gender recognition rates can be achieved in the same face/gait dataset, they assume a closed-world and neglect the problems caused by dataset bias. Real-world human gender recognition system should be dataset-independent, i.e., it can be trained on one face/gait dataset and tested on another. In this paper, the authors test several popular face/gait-based gender recognition algorithms in a cross-dataset manner. The recognition rates decrease significantly and some of them are only slightly better than random guess. These observations suggest that the generalization power of conventional algorithms is less satisfied, and highlight the need for further research on face/gait-based gender recognition for real-world applications.

CROSS-VIEW GAIT RECOGNITION WITH SHORT PROBE SEQUENCES: FROM VIEW TRANSFORMATION MODEL TO VIEW-INDEPENDENT STANCE-INDEPENDENT IDENTITY VECTOR

Considering it is difficult to guarantee that at least one continuous complete gait cycle is captured in real applications, we address the multi-view gait recognition problem with short probe sequences. With unified multi-view population hidden markov models (umvpHMMs), the gait pattern is represented as fixed-length multi-view stances. By incorporating the multi-stance dynamics, the well-known view transformation model (VTM) is extended into a multi-linear projection model in a four-order tensor space, so that a view-independent stance-independent identity vector (VSIV) can be extracted. The main advantage is that the proposed VSIV is stable for each subject regardless of the camera location or the sequence length. Experiments show that our algorithm achieves encouraging performance for cross-view gait recognition even with short probe sequences.

Incremental Learning for Video-Based Gait Recognition With LBP Flow

Gait analysis provides a feasible approach for identification in intelligent video surveillance. However, the effectiveness of the dominant silhouette-based approaches is overly dependent upon background subtraction. In this paper, we propose a novel incremental framework based on optical flow, including dynamics learning, pattern retrieval, and recognition. It can greatly improve the usability of gait traits in video surveillance applications. Local binary pattern (LBP) is employed to describe the texture information of optical flow. This representation is called LBP flow, which performs well as a static representation of gait movement. Dynamics within and among gait stances becomes the key consideration for multiframe detection and tracking, which is quite different from existing approaches. To simulate the natural way of knowledge acquisition, an individual hidden Markov model (HMM) representing the gait dynamics of a single subject incrementally evolves from a population model that reflects the average motion process of human gait. It is beneficial for both tracking and recognition and makes the training process of the HMM more robust to noise. Extensive experiments on widely adopted databases have been carried out to show that our proposed approach achieves excellent performance.

Discriminant learning through multiple principal angles for visual recognition

Canonical correlation has been prevalent for multiset-based pairwise subspace analysis. As an extension, discriminant canonical correlations (DCCs) have been developed for classification purpose by learning a global subspace based on Fisher discriminant modeling of pairwise subspaces. However, the discriminative power of DCCs is not optimal as it only measures the "local" canonical correlations within subspace pairs, which lacks the "global" measurement among all the subspaces. In this paper, we propose a multiset discriminant canonical correlation method, i.e., multiple principal angle (MPA). It jointly considers both "local" and "global" canonical correlations by iteratively learning multiple subspaces (one for each set) as well as a global discriminative subspace, on which the angle among multiple subspaces of the same class is minimized while that of different classes is maximized. The proposed computational solution is guaranteed to be convergent with much faster converging speed than DCC. Extensive experiments on pattern recognition applications demonstrate the superior performance of MPA compared to existing subspace learning methods.